Alex Plugaru2023-10-12T20:03:48.517Zhttps://plugaru.org/Alex PlugaruHexoScaling engineering at Gorgiashttps://plugaru.org/2023/08/17/unblocked-podcast/2023-08-17T11:20:00.000Z2023-10-12T20:03:48.517Z
<p>A talk about how we scaled our engineering team at Gorgias, cognitive load theory for organizing teams, when to split a team, what to
Big things have small beginningshttps://plugaru.org/2023/03/29/big-things-small-begginings/2023-03-29T11:33:25.000Z2023-10-12T20:03:48.517Z
<p><img src="/images/bigsmall/first-line-of-code.jpg" alt="first commit gorgias" title="first line of code at Gorgias"></p>
<p>Ten years
Gorgias's history, mission and what's nexthttps://plugaru.org/2022/07/28/dtc-pod-podcast/2022-07-28T11:20:00.000Z2023-10-12T20:03:48.517Z
<p>Big thanks Ramon Berrios & Blaine Bolus for inviting me to talk about Gorgias’s history, it’s mission and what is coming next.
Engineering management bookshttps://plugaru.org/2022/04/30/engineering-management-books/2022-04-30T11:20:00.000Z2023-10-12T20:03:48.517Z
<p>If you’re like me (first time manager in a <a href="https://www.gorgias.com" target="_blank" rel="noopener">fast growing software
A Thousand Brains - book review & summary.https://plugaru.org/2021/03/10/a-thousand-brains/2021-03-10T12:20:00.000Z2023-10-12T20:03:48.517Z
<p>The <a href="https://en.wikipedia.org/wiki/Neocortex" target="_blank" rel="noopener">neocortex</a>: the organ that occupies ~80% volume
PyCharm Tip: Show only tableshttps://plugaru.org/2018/11/22/PyCharm-Tip-Show-only-tables/2018-11-22T19:00:17.000Z2023-10-12T20:03:48.517Z
<p>If you work with PyCharm or DataGrip with a database and you want to access the table data it can be pretty annoying to get to them
Introducing kuku: kubernetes template toolhttps://plugaru.org/2018/09/27/kuku/2018-09-27T21:54:02.000Z2023-10-12T20:03:48.517Z
<p>At <a href="https://gorgias.io/" target="_blank" rel="noopener">Gorgias</a> we’re using <a href="https://kubernetes.io/" target="_blank"
Managing DNS zones with GCP deployment managerhttps://plugaru.org/2018/08/21/managing-dns-zones-with-gcp-deployment-manager/2018-08-21T21:54:02.000Z2023-10-12T20:03:48.517Z
<p>I looked for an example of how to setup a managed DNS zone on GCP (using their deployment manager) with a Global IP address to no
Pod sandbox changed, it will be killed and re-created.https://plugaru.org/2018/05/21/pod-sandbox-changed-it-will-be-killed-and-re-created/2018-05-21T15:46:48.000Z2023-10-12T20:03:48.517Z
<p>It all started at 6PM Friday night when we were in the middle of our new office warming party - as any admin can tell you it’s best time
Highly available Celery with RabbitMQ and Kuberneteshttps://plugaru.org/2018/01/03/rabbitmq-celery-kubernetes-ha/2018-01-03T00:52:03.000Z2023-10-12T20:03:48.517Z
<p><a href="https://kubernetes.io/" target="_blank" rel="noopener">Kubernetes</a>, <a href="https://www.rabbitmq.com/" target="_blank"
PostgreSQL backup with pghoard & Kuberneteshttps://plugaru.org/2016/07/17/postgresql-backup-with-pghoard-kubernetes/2016-07-17T23:50:33.000Z2023-10-12T20:03:48.517Z
<p><strong>TLDR:</strong> <a href="https://github.com/xarg/pghoard-k8s" target="_blank"
My very subjective future of humanity and strong* AIhttps://plugaru.org/2016/04/24/future-of-humanity-and-strong-AI/2016-04-24T23:06:08.000Z2023-10-12T20:03:48.517Z
<p>The fascination with <a href="https://en.wikipedia.org/wiki/Artificial_general_intelligence" target="_blank" rel="noopener">AGI</a> has been mainstream for a long time, but it started having more even more momentum in the recent years. Even hollywood has become less naive with movies like <a href="http://www.imdb.com/title/tt1798709" target="_blank" rel="noopener">Her</a> and <a href="http://www.imdb.com/title/tt0470752/" target="_blank" rel="noopener">Ex Machina</a>.</p>
<p>On the R&D side there is of course Deep Learning which is a machine learning technique that uses neural networks with 1 hidden layer :P It has changed I believe forever the way people are doing research today. The hype is real because of the state of the art results achieved with it and the way the skills translate across different fields of ML. AlphaGo beats the best player in the world, translation and image/voice recognition is becoming better, artistic style stealing, attention models, etc.. The best part is that it’s more or less the same RNN with different neuron architectures, backprop and gradient decent that works with a broad range of problems. Now people are looking to for nails because they have a damn mighty hammer.</p>
<p>Of course hooking up a bunch of NVidia Pascals is not gonna give us AGI and the Moore’s law is not what it used to be. I could not agree more, but if we overcome the hardware issues (and I have high hopes that AR and VR is gonna push this) then it’s reasonable to assume that we’ll have the hardware to achieve at least weak AI soonish…</p>
<p>What about software? That maybe a bigger problem. But.. I’m also optimistic here with things like torch and recently tensorflow are given ton of attention from one of the best minds in the AI world today. What’s really cool about these frameworks is that they are used everyday in production on real products by startups and big corp alike. They are here to stay. It’s not enough, but I’m hopeful that things will improve.</p>
<p>Ok, so I want to say something that has been bugging me a long time, bare with me, I believe it’s important for the arguments that follow. </p>
<blockquote><p>… is the intelligence of a (hypothetical) machine that could successfully perform any intellectual task that a human being can…</p>
<footer><strong>Wikipedia</strong><cite><a href="https://en.wikipedia.org/wiki/Artificial_general_intelligence" target="_blank" rel="noopener">Artificial General Intelligence</a></cite></footer></blockquote>
<p>Now I have a problem with this definition because I would argue that in a cosmic sense we, the humans, haven’t achieved what I would call general intelligence. We’re kind of good at surviving in the Earth’s atmosphere. We can do many things that are amazing and not accessible to most animals, but we’re still bound to our environment. We’re still I would argue narrow in our intelligence and can only grasp a small fraction of what’s out there.
There exists true AGI which is AIXI. It will seek to maximize its future reward in any computable environment (survive and expand), but there is this tiny little problem of requiring infinite memory and computing power in order for it to function. It’s useful just like the Turing machine is useful in the real world.
For any intelligent agent to be practical, it’s required a favourable environment and a narrow specialisation for that environment. This is why I think that we’re really after is strongish AI which translates to being pretty cool in your neighbourhood.</p>
Running Flask & Celery with Kuberneteshttps://plugaru.org/2016/03/04/flask-celery-kubernetes/2016-03-04T00:49:31.000Z2023-10-12T20:03:48.517Z
<p>At <a href="https://gorgias.io" target="_blank" rel="noopener">Gorgias</a> we recently switched our <a href="http://flask.pocoo.org" target="_blank" rel="noopener">flask</a> & <a href="http://www.celeryproject.org/" target="_blank" rel="noopener">celery</a> apps from Google Cloud VMs provisioned with <a href="http://www.fabfile.org/" target="_blank" rel="noopener">Fabric</a> to using <a href="https://www.docker.com/" target="_blank" rel="noopener">docker</a> with <a href="http://kubernetes.io/" target="_blank" rel="noopener">kubernetes</a> (k8s). This is a post about our experience doing this.</p>
<p><strong>Note:</strong> I’m assuming that you’re somewhat familiar with Docker.</p>
<h2 id="Docker-structure"><a href="#Docker-structure" class="headerlink" title="Docker structure"></a>Docker structure</h2><p>The killer feature of Docker for us is that it allows us to make layered binary images of our app. What this means is that you can start with a minimal base image, then make a python image on top of that, then an app image on top of the python one, etc..</p>
<p>Here’s the hierarchy of our docker images:</p>
<ul>
<li><a href="https://github.com/mbentley/dockerfiles/tree/master/ubuntu/pgbouncer" target="_blank" rel="noopener">gorgias/pgbouncer</a></li>
<li><a href="https://gist.github.com/xarg/7fe2af07ebd7a4b9bdca" target="_blank" rel="noopener">gorgias/rabbitmq</a></li>
<li>gorgias/nginx <em>- extends gorgias/base and installs NGINX</em></li>
<li>gorgias/python3 <em>- Installs pip, python3.5 - yes, using it in production.</em><ul>
<li>gorgias/app <em>- This installs all the system dependencies: libpq, libxml, etc.. and then does pip install -r requirements.txt</em><ul>
<li>gorgias/web <em>- this sets up uWSGI and runs our flask app</em></li>
<li>gorgias/worker <em>- Celery worker</em></li>
</ul>
</li>
</ul>
</li>
</ul>
<p><strong>Piece of advice:</strong> If you used to run your app using supervisord before I would advise to avoid the temptation to do the same with docker, just let your container crash and let your Kubernetes/Swarm/Mesos handle it.</p>
<p>Now we can run the above images using: <a href="https://docs.docker.com/compose/" target="_blank" rel="noopener">docker-compose</a>, <a href="https://docs.docker.com/swarm/overview/" target="_blank" rel="noopener">docker-swarm</a>, <a href="http://kubernetes.io/" target="_blank" rel="noopener">k8s</a>, <a href="https://mesosphere.com/" target="_blank" rel="noopener">Mesos</a>, etc…</p>
The 100% software companyhttps://plugaru.org/2015/07/21/100-software-company/2015-07-21T23:47:06.000Z2023-10-12T20:03:48.517Z
<p>If you know <a href="https://stripe.com" target="_blank" rel="noopener">Stripe</a>, <a href="https://www.mailgun.com/" target="_blank"
Near-future is for human-computer hybridshttps://plugaru.org/2015/01/02/near-future-human-computer-hybrids/2015-01-02T18:45:20.000Z2023-10-12T20:03:48.517Z
<p><img src="/images/magician.jpg" alt="customer support oracle" title="customer support oracle"> </p>
<p>Most of tech startups today try
How much energy is needed to write an e-mail?https://plugaru.org/2014/11/26/energy-needed-to-write-an-e-mail/2014-11-26T00:43:06.000Z2023-10-12T20:03:48.517Z
<p><strong>TL;DR;</strong> </p>
<p>In 2013 we sent <strong><a href="http://www.radicati.com/wp/wp-content/uploads/2013/09/Email_Market_2013-2017_Executive_Summary.pdf" target="_blank" rel="noopener">182.9B</a></strong> emails, consuming <code>26PJ</code> (Peta Joules). Equivalent of yearly electricity consumed in Mongolia and 1kg of <a href="http://en.wikipedia.org/wiki/Mass-energy" target="_blank" rel="noopener">matter-energy</a>.</p>
<p>Every time an e-mail is written, or any sort of human-computer interaction, some amount of energy is expended. Given that in 2013 an estimate of <strong><a href="http://www.radicati.com/wp/wp-content/uploads/2013/09/Email_Market_2013-2017_Executive_Summary.pdf" target="_blank" rel="noopener">182.9 billion</a></strong> e-mails written world wide an interesting question pops up: </p>
<h3 id="How-much-energy-is-needed-to-write-an-e-mail"><a href="#How-much-energy-is-needed-to-write-an-e-mail" class="headerlink" title="How much energy is needed to write an e-mail?"></a>How much energy is needed to write an e-mail?</h3><p>By <strong>energy</strong> I mean how many <code>Joules</code> and by <strong>needed</strong> I mean needed by the human who writes the e-mail and the computer used to write the respective e-mail. For our calculations we’ll take an <strong>e-mail</strong> length of a 100 words. I don’t know the average e-mail size, but perhaps I should ask NSA, FSB or MSS some time to get a more accurate idea.</p>
<blockquote><p>Sending an e-mail involves e-mail servers, routers, switches and other network devices which require energy in order to function, but given that the topology of the internet is rather hard to take into account, we’ll not bother with these details. However I suspect that the infrastructure itself that is required to send a single e-mail consumes far more energy that opening up your computer and sending the e-mail. </p>
</blockquote>
<p>At this moment I’m not sure if it’s even possible to calculate this accurately, but let’s try anyway and please if you find any mistakes feel free to point them out. </p>
zope.component introductionhttps://plugaru.org/2011/11/24/zope-component-introduction/2011-11-24T01:28:48.000Z2023-10-12T20:03:48.517Z
<p>Recently I gave a talk about zope component architecture at a company in Paris where I started consulting on zope, plone, django and python in general. In these posts, aided by code examples, I will try to show what zope components are capable of, what are the common use-cases and some of my personal views about the subject.</p>
<p>First lets identify the problem that we are trying to solve using a component architecture:</p>
<p>Applications tend to get big. Especially those applications that have a lot of users and have years of development. Users want different features/changes in different stages of the development of the product. Hence the key to a successful and long lasting application is to comply with the user needs as fast as possible with as little technical dept as possible. Finding a balance between the two is not easy.</p>
<p>Speed of development is important especially in the beginning of a project. Your clients want that software today and you can’t waste time on thinking a lot about long term benefits if your bank account is empty. Those benefits are nonexistent if you don’t finish in time so the shortest path usually gives you the best chances of success. This is the correct way to think about it. However when the project is getting bigger you may want to start thinking in different terms. Questions like these are usually good ones to ask:</p>
<ul>
<li>How much time does it take to create a feature?</li>
<li>Refactoring is hard, but can it be made faster?</li>
<li>Is it easy to extend existing features?</li>
<li>How different parts of your application talk to each other?</li>
<li>How about debugging?</li>
</ul>
<p>These questions matter both to managers and coders. Even if the managers and developer are very good at what they do usually what happens is that layer after layer gets added on top of the existing product until it becomes a big pile of crap – new developers are screwed because they can’t figure out what is happening in the system and the time to train them is very expensive. New features take enormous amounts of time to develop and your technical dept gets bigger and bigger with every new modification.</p>
<p>Very little can be done about this once you’re big. There are no easy solutions and old applications sometimes just whither and die. Tough. But maybe components can help.</p>