Siddhant Goel

Self-hosting static sites on a Caddy-powered VPS

2026-06-02T00:00:00+00:00

I operate a few different static websites, relatively low-traffic ones. So far these sites were hosted on managed hosting providers, including Netlify, Cloudflare Pages, and Bunny CDN. Last weekend I moved most of them over to an Ubuntu Linux VPS, where they're being served using Caddy</a>.

In this post I'll walk through why I made the switch, how the new setup works, what I like about it, and how Caddy makes hosting multiple domains from one server surprisingly painless.

Why move away from managed hosting</h2>
Managed hosting is a fine way to serve websites, especially if the contents are static. For the longest time, Netlify was my go-to option for serving static sites. Over time though, I started getting bothered by the fact that I was adapting more and more of my workflows according to how the provider worked instead of what I actually needed.
This was visible in mainly two aspects - configuration and deployment workflows.
Every hosting provider has their own way of setting configuration. If you need to set a specific HTTP header, for instance, you either define it in their user interface somewhere or put it in a configuration file. Netlify, for instance, has a `netlify.toml</code> where all this stuff can go. Maybe it's just because I'm older now, but over time I've come to the realization that I wantless configuration</a> in my life, not more. I want things to just work, and the less custom configuration there is the better.`
`Deployment workflows were a similar story. When setting up automatic deployments I need to allow access to the Github/Gitlab repository. One of my static sites was hosted on Bunny CDN backed by a pull zone and a storage bucket. Since Bunny doesn't provide a native way to deploy static sites, I had Claude vibe-code a Python script that would copy files from the local disk to the bucket and then manually purge the cache. It worked fine, but it was yet another thing for me to maintain.`

Setting up the VPS</h2>
In my mind, the only logical solution was to set up a Linux VPS and put those static
sites behind a proxy server. I've operated Linux servers in the past, and self-hosting
static websites is really not a lot of work. It'll still involve configuring bits and
pieces, but at least the configuration would not be tied to a hosting provider.</p>
So I looked around for cheap VPS providers, ideally based in the EU, and came across
Netcup</a>. They offered a plan costing €3 per month for a server with 2 CPU cores, 2GB
RAM, and 60GB SSD, which sounded more than enough for what I needed, so I provisioned a
VPS on that plan and got to work.</p>
The first step was to generate an SSH key and copy it over to the VPS for password-less
login. Next, I set up a few infrastructure definitions using pyinfra</a> so that all the
changes I make to the server are contained in source files somewhere.</p>
Setting up pyinfra inventory</h2>
When using pyinfra, the host configuration is usually put in something called an
"inventory", that defines the IP address of the VPS along with the SSH configuration
needed to log in to it.</p>
# inventory.py</span></span>
</span>
import</span> os.path</span></span>
</span>
servers</span> =</span> [</span></span>
    (</span></span>
        "1.2.3.4"</span>,</span></span>
        {</span></span>
            "ssh_user"</span>:</span> "username"</span>,</span></span>
            "ssh_key"</span>: os.path.expanduser(</span>"~/.ssh/id_rsa"</span>),</span></span>
            "_sudo"</span>:</span> True</span>,</span></span>
        },</span></span>
    ),</span></span>
]</span></span></code></pre>
It's basically just a Python list of servers where each entry is a tuple containing the
server's IP address and SSH arguments. With this inventory in place, pyinfra is able to
log in to the VPS on our behalf to perform actions.</p>
Setting up pyinfra Operations</h2>
The next step after configuring a pyinfra inventory is to define a few operations.
Operations are how we can instruct pyinfra to take actions and make changes on the
target machines.</p>
Below is the complete deploy.py</code> with all operations. Each section is explained
afterward.</p>
# deploy.py</span></span>
</span>
from</span> pyinfra.operations</span> import</span> apt, files, server, systemd</span></span>
</span>
apt.packages(</span></span>
    name</span>=</span>"Ensure base packages"</span>,</span> packages</span>=</span>[</span>"caddy"</span>,</span> "rsync"</span>],</span> update</span>=</span>True</span></span>
)</span></span>
</span>
server.shell(</span></span>
    name</span>=</span>"Set up ufw"</span>,</span></span>
    commands</span>=</span>[</span></span>
        "ufw default deny incoming"</span>,</span></span>
        "ufw default allow outgoing"</span>,</span></span>
        "ufw allow 22/tcp comment 'SSH'"</span>,</span></span>
        "ufw allow 80/tcp comment 'HTTP'"</span>,</span></span>
        "ufw allow 443/tcp comment 'HTTPS'"</span>,</span></span>
        "ufw enable"</span>,</span></span>
    ],</span></span>
)</span></span>
</span>
files.directory(</span></span>
    path</span>=f</span>"/srv/example.com"</span>,</span></span>
    present</span>=</span>True</span>,</span></span>
    recursive</span>=</span>True</span>,</span></span>
    user</span>=</span>"caddy"</span>,</span></span>
    group</span>=</span>"caddy"</span>,</span></span>
)</span></span>
</span>
caddyfile</span> =</span> files.template(</span></span>
    name</span>=</span>"Update Caddy configuration"</span>,</span></span>
    src</span>=</span>"Caddyfile"</span>,</span></span>
    dest</span>=</span>"/etc/caddy/Caddyfile"</span>,</span></span>
)</span></span>
</span>
systemd.service(</span></span>
    name</span>=</span>"Start and enable Caddy service"</span>,</span></span>
    service</span>=</span>"caddy.service"</span>,</span></span>
    running</span>=</span>True</span>,</span></span>
    enabled</span>=</span>True</span>,</span></span>
    restarted</span>=</span>caddyfile.will_change,</span></span>
    daemon_reload</span>=</span>caddyfile.will_change,</span></span>
)</span></span></code></pre>
Here's what each section does.</p>
Setting up a firewall</h3>
The server.shell</code> block configures ufw</code> to deny all incoming traffic by default and
allow outgoing traffic, then opens ports 22 (SSH), 80 (HTTP), and 443 (HTTPS).</p>
pyinfra currently does not have a native operation for ufw</code>, the rationale being that
ufw</code> rules ultimately translate to operations.iptables</code> rules. I tried using
iptables</code> but ran into a bunch of errors, so I decided to just call ufw</code> via
operations.server</code> and move on.</p>
Beyond the firewall, it may be worth spending a bit more time on hardening SSH access.
Things like disabling password authentication and optionally changing the default port
would significantly reduce brute-force attempts.</p>
Installing Caddy and hosting multiple domains</h3>
After installing Caddy, it's likely that Caddy's default configuration is not what you
want since you probably have your own websites you'd like to deploy.</p>
Serving static sites is quite straightforward using Caddy. Assuming that the contents of
your static website (HTML, CSS, JS, etc.) is at /srv/example.com</code>, here is a minimal
(but fully functional) Caddy configuration:</p>
example.com {</span></span>
    root * /srv/example.com</span></span>
    encode gzip</span></span>
    file_server</span></span>
</span>
    log {</span></span>
        format console</span></span>
        output file /var/log/caddy/example.com.log {</span></span>
            roll_size 10MB</span></span>
        }</span></span>
    }</span></span>
}</span></span>
</span>
www.example.com {</span></span>
    redir https://example.com{uri} permanent</span></span>
}</span></span></code></pre>
Copy the contents of the snippet above into a local file (let's name it Caddyfile</code>)
next to inventory.py</code> and deploy.py</code>.</p>
Looking back at the deploy.py</code> script, the apt.packages</code> call installs Caddy and
rsync (we'll need rsync later for deployments). The files.directory</code> call creates
/srv/example.com</code> and makes
sure that this directory is owned by the caddy</code> user and group. This is where the
contents of our static site would go, and the correct user/group permissions are
important for Caddy to be able to read (and serve) files from this directory.</p>
Next, files.template</code> copies the Caddyfile from local disk to
/etc/caddy/Caddyfile</code>, and systemd.service</code> runs and enables the Caddy service,
also making sure that whenever the contents of Caddyfile</code> change, the associated
systemd service and the systemd daemon itself is restarted.</p>
One thing to keep in mind: make sure your domain's DNS records (an A record for the apex
and a CNAME for www</code>) point to the VPS IP address before starting Caddy. Caddy needs
the domain to resolve before it can provision TLS certificates automatically.</p>
Deployment Workflow</h2>
After all this is in place, the last remaining piece is how to deploy your website. The
contents of the site will be deployed to /srv/example.com</code> on the VPS, so the
"deployment" process is nothing more than copying a bunch of files from the local disk
to the VPS, which is exactly where rsync</code> shines!</p>
There are a few different ways to use rsync</code> to copy contents over. I've personally
defined a deploy-site</code> bash script that looks as follows:</p>
#!/usr/bin/env bash</span></span>
</span>
set -e</span></span>
</span>
if</span> [</span> $#</span> -ne</span> 2</span> ];</span> then</span></span>
    echo</span> "Usage: $(</span>basename</span> "</span>$0</span>") <source_dir> <domain>"</span> >&2</span></span>
    exit 1</span></span>
fi</span></span>
</span>
if</span> [</span> ! -d</span> "</span>$1</span>"</span> ];</span> then</span></span>
    echo</span> "Error: source directory '</span>$1</span>' does not exist"</span> >&2</span></span>
    exit 1</span></span>
fi</span></span>
</span>
rsync</span> -avz --progress --rsync-path</span> "sudo -u caddy rsync" "</span>${1</span>%/</span>}</span>/" "vps:/srv/</span>$2</span>"</span></span></code></pre>
and placed it at $HOME/.local/bin/deploy-site</code>. Whenever I start a new project,
independent of how the final HTML/CSS/... artifacts are generated, I call this script
like deploy-site dist/ example.com</code>, that calls rsync, and within just a few seconds
the site is deployed.</p>
Note that the --rsync-path "sudo -u caddy rsync"</code> flag requires the remote user to have
passwordless sudo, which you'll need to set up if you haven't done so already.</p>
Conclusion</h2>
All in all, the migration took less than a day and the setup has been running without
issues since. The next thing I'd like to add is some form of monitoring, for instance
Prometheus metrics to keep track of CPU and memory usage, just for fun. Maybe that's
what the next post will be about!</p>


What's next for txtcv
2026-05-10T00:00:00+00:00
A few months ago I started</a> working on a CV builder for software developers. It has now
seen some usage beyond just me, and I've learned a few things over the course. This post
covers what I'm planning for txtcv in the near future.</p>
From CLI powered to CLI-first</h2>
The first change I'd like to make is for the project to be more CLI-driven.</p>
At the moment there is</em> a CLI, but it's more of a frontend rather than being the
application itself. If you sign up on txtcv.com, your CV and your job applications and
everything else is saved on the server. This choice made sense in the beginning, but the
more I work with it the more it bothers me. I'd like to give the user more control and
let them store all the data locally, on their disk. The most straightforward way of
doing that is to make the CLI front and center.</p>
This doesn't mean that I'd like to get rid of the backend. Running something on the
server lets me build features that aren't possible just with a CLI (e.g. generating
cover letters using an LLM). Plus, I'd still like to be able to offer a paid tier, and
keeping track of license keys is a lot more ergonomic on the backend. Regardless, I
imagine this backend to provide optional features, while the majority of the application
runs on the user's command line, in their control.</p>
Open-source core</h2>
Having a command-line app means the user also expects to be able to read the source
code. I personally feel a bit uneasy about installing binaries on my machine without
knowing what they do, so I imagine there are others that feel that way too.</p>
So, I'd like for this CLI to be open-source. Given GitHub's recent stability problems, I
decided to go with Tangled for this repository, and the source code will be accessible
at https://tangled.org/txtcv.com/cli</a>. Tangled, in case you don't know, is a Git
hosting service that utilizes the AT protocol</a> extensively.</p>
The repository doesn't contain much at the time of this writing, because it's still
very early days for this change. But expect that to change over the next weeks!</p>
Learning Go</h2>
Python has been my main programming language for more than a decade now. Recently, I've
been itching to experiment with another programming language. And rebuilding this
project, I believe, provides me with the perfect opportunity to learn something new.</p>
Like many other software developers, I've been looking at Go and Rust recently. For this
project, I wanted to use a language that makes it easy to write command-line
applications and has a good deployment story. While both Go and Rust fit the bill, I
ultimately decided to pick Go because it's a lot</em> easier to learn as compared to Rust.</p>
I spent some time trying to learn Rust, but it's a massive language with a very steep
learning curve, which is a big barrier to get past before I can do anything useful with
it. In comparison, Go was much easier to get started with (I spent a weekend working
through the Tour of Go</a> and Go by Example</a>), and I was able to write useful code a lot
quicker. So, over the next few weeks I'd like to build out the application more and
more, and learn Go in the process.</p>

That's the direction I'd like to take txtcv in: a CLI-first, open-source tool that gives
you full control over your data, with optional server-side features for the things that
benefit from it. And I get to learn a new language along the way, which is always fun.</p>


PostgreSQL on Kamal: Deployment, Configuration, and Backups
2026-03-01T00:00:00+00:00
It's been a few weeks since I started building txtcv</a>, which is a simple career
management platform (CV builder, cover letter generator, job application tracker) aimed
mostly at software developers. Built using Django and some Htmx (where needed), it's
deployed on a 2.50 €/month machine on a VPS hosting provider using Kamal.</p>
When starting out, I consciously made the decision to not use any cloud hosting
providers, mainly because I wanted a side-project to be fun</strong> to work on (and cloud
hosting providers take the fun out of deployments if you're the kind of person who has
fun when deploying applications), but also because you can get a lot</strong> more processing
power on a VPS than on a cloud hosting provider for the same amount of money.</p>
After looking around and experimenting with tools that would make this process easy, I
settled on Kamal.</p>
Kamal Deploy</h2>
Kamal, if you're not familiar, is a command-line tool that lets you easily deploy web
applications to servers. Apart from deployments, it also handles tasks like setting up
backing services (e.g. database containers), automated SSL certificate provisioning and
renewal, zero-downtime deploys, etc.</p>
The setup involves specifying a bunch of configuration in a YAML file – things like your
server's IP address, where to fetch your application's Docker image from, environment
variable definitions, etc. Once the setup is done, you can run kamal deploy</code> and have
an updated version of your application running on your server within just a few minutes.
And if your application requires a backing service (such as a PostgreSQL database),
Kamal lets you define "accessories" in the configuration file that are ultimately
containers that run alongside your main web application, providing a specific service.
So, for instance, if you're running a Django application that requires PostgreSQL, you
could define a postgres</code> accessory that your Django application can connect to.</p>
Once you have the workflow set up, it's really quite painless.</p>
Defining a PostgreSQL accessory</h2>
The previous section handwaved accessories, but here's an example YAML snippet that
hopefully makes the PostgreSQL setup in a Kamal context a bit more tangible.</p>
accessories</span>:</span></span>
  postgres</span>:</span></span>
    image</span>:</span> postgres:18</span></span>
    host</span>:</span> 1.2.3.4</span></span>
    port</span>:</span> "127.0.0.1:5432:5432"</span></span>
    env</span>:</span></span>
      clear</span>:</span></span>
        POSTGRES_DB</span>:</span> example</span></span>
        POSTGRES_USER</span>:</span> example</span></span>
      secret</span>:</span></span>
        -</span> POSTGRES_PASSWORD</span></span>
    directories</span>:</span></span>
      -</span> data:/var/lib/postgresql</span></span></code></pre>
This snippet instructs Kamal to fetch the postgres:18</code> Docker image from the Docker hub
and run a PostgreSQL container alongside your main web application. The environment
variables defined under the env</code> key will be passed on to the postgres</code> container and
the port/directory mapping will be done as defined in the configuration.</p>
Backing up PostgreSQL to Ionos Storage</h2>
If you're running a database you should also consider backing it up somewhere. With
PostgreSQL, one simple and easy way is to run pg_dump</code> on your database and store the
result on an S3 bucket.</p>
In the Kamal world, we can use an accessory to do this. The
siemens/postgres-backup-to-s3</a> project provides a Docker image to periodically back up
a PostgreSQL database to Amazon S3. We can reference that image to define and boot an
accessory, that, once running, will perform automated database backups for us. It's
meant for S3 buckets, but if the object storage provider of your choice is S3
compatible, that should work just as well.</p>
Here's an example YAML snippet that shows how this can work in practice:</p>
accessories</span>:</span></span>
  postgres-backup</span>:</span></span>
    image</span>:</span> siemens/postgres-backup-s3:18</span></span>
    host</span>:</span> 1.2.3.4</span></span>
    env</span>:</span></span>
      clear</span>:</span></span>
        SCHEDULE</span>:</span> "@daily"</span></span>
        BACKUP_KEEP_DAYS</span>:</span> 7</span></span>
        S3_ENDPOINT</span>:</span> https://s3.eu-central-3.ionoscloud.com</span></span>
        S3_REGION</span>:</span> eu-central-3</span></span>
        S3_BUCKET</span>:</span> example</span></span>
        S3_PREFIX</span>:</span> backups</span></span>
        POSTGRES_HOST</span>:</span> example-postgres</span></span>
        POSTGRES_DATABASE</span>:</span> example</span></span>
        POSTGRES_USER</span>:</span> example</span></span>
      secret</span>:</span></span>
        -</span> POSTGRES_PASSWORD</span></span>
        -</span> S3_ACCESS_KEY_ID</span></span>
        -</span> S3_SECRET_ACCESS_KEY</span></span></code></pre>
We define a postgres-backup</code> accessory that uses the siemens/postgres-backup-to-s3</code>
Docker image. The environment variables are all meant for the running container,
defining the backup schedule, the S3 credentials, and details of the database that
should be backed up. This piece of configuration will make sure that our database is
backed up every night and the resulting file is storegd on the configured Ionos storage
bucket.</p>
Note that the S3_ENDPOINT</code> variable does not have anything to do with Amazon S3,
instead referencing an ionoscloud</code> domain name. Ionos storage buckets are S3
compatible, meaning that if you have some code meant to run with S3, you can basically
swap out the bucket URL and the S3 credentials and it should all just continue to work.</p>
Conclusion</h2>
After using Kamal over the past few months, I've come to really like it. It takes away a
lot of the grunt work associated with deploying user-facing web applications. The setup
I described above has been running reliably for my side project, handling both the main
database and the nighly backups without intervention. In case you're considering Kamal
for your next deployment, I'd definitely recommend!</p>


10 years of personal finances in plain text files
2025-12-20T00:00:00+00:00
January 2026 will mark 10 years since I started storing my personal finances in plain
text files using Beancount. Since January 2016, I've taken out about 30-45 minutes every
single month to download my monthly bank statements and import them into my Beancount
ledger.</p>
There's a lot to talk about here, but let's start with some fun numbers!</p>
The 10 year old Beancount ledger</h2>
10 years of financial transactions is a lot of data! All in all, my ledger contains over
45,000 lines of Beancount entries spread across 16 plain text files. All of it is stored
in a finances</code> directory (version controlled) on my laptop. Here's a snapshot:</p>
❯</span> find .</span> -name</span> "*.beancount"</span> |</span> xargs</span> wc</span> -l</span></span>
    4037</span> ./includes/2020.beancount</span></span>
    3887</span> ./includes/2018.beancount</span></span>
      27</span> ./includes/cash.beancount</span></span>
    4398</span> ./includes/2021.beancount</span></span>
    5531</span> ./includes/2019.beancount</span></span>
    5267</span> ./includes/2022.beancount</span></span>
    3287</span> ./includes/2017.beancount</span></span>
    5506</span> ./includes/2024.beancount</span></span>
    5606</span> ./includes/2023.beancount</span></span>
    1454</span> ./includes/2016.beancount</span></span>
    1089</span> ./includes/open/04-expenses.beancount</span></span>
      66</span> ./includes/open/03-income.beancount</span></span>
      11</span> ./includes/open/05-liabilities.beancount</span></span>
      37</span> ./includes/open/02-assets.beancount</span></span>
       1</span> ./includes/open/01-equity.beancount</span></span>
    4807</span> ./main.beancount</span></span>
   45011</span> total</span></span></code></pre>
Running bean-query</code> on main.beancount</code> tells me I have about 10,000 transactions in
total, that in turn contain about 20,000 postings (in double-entry bookkeeping, one
transaction may have multiple postings).</p>
❯</span> uv run bean-query main.beancount</span></span>
Input</span> file: "Goel"</span></span>
Ready</span> with</span> 12466</span> directives</span> (19743</span> postings in</span> 9895</span> transactions</span>).</span></span>
beanquery</span>></span></span></code></pre>
There are 1086 accounts in total.</p>
beanquery</span>></span> select count</span>(</span>*</span>)</span> from</span> (select</span> distinct</span>(</span>account</span>));</span></span>
coun</span></span>
----</span></span>
1086</span></span></code></pre>
... which does not mean that there are 1086 bank</strong> accounts. Accounts in Beancount are
virtual, and you can create as many as you like. Imagine one account for categorizing
supermarket spending, one for tracking your income, one for your Netflix subscription,
and so on.</p>
Next, there are about 500 documents in the repository.</p>
❯</span> find documents/</span> -name</span> "*.pdf"</span> |</span> wc</span> -l</span></span>
     507</span></span></code></pre>
Beancount lets you attach documents (e.g. receipts or invoices) to transactions, that
makes bookkeeping very efficient. I love the fact that whenever I need to do my tax
returns, I can just take a look at my Beancount ledger and find all the invoices right
there next to the relevant transaction.</p>
Lastly, in terms of postings, I started out with 715 in the year 2016, and the year 2023
was the busiest so far in terms of just the total postings count.</p>
beanquery</span>></span> select year</span>(</span>date</span>)</span>, count</span>(</span>*</span>)</span> where year</span>(</span>date</span>)</span> <</span> 2025</span> group by year</span>(</span>date</span>);</span></span>
year</span>  coun</span></span>
----</span>  ----</span></span>
2016</span>   715</span></span>
2017</span>  1422</span></span>
2018</span>  1605</span></span>
2019</span>  2437</span></span>
2020</span>  1582</span></span>
2021</span>  2022</span></span>
2022</span>  2435</span></span>
2023</span>  2651</span></span>
2024</span>  2602</span></span></code></pre>The Monthly Ritual</h2>
I wrote earlier that every month I take about 30-45 minutes to import my financial
transactions into Beancount. What does that workflow look like? I wrote another, much
more detailed blog post</a> about it a few years ago, but here's a gist.</p>
It starts with me logging in to my bank account(s) to download my monthly statement(s)
in CSV (CSV because it's much more predictable to parse compared to PDF). I then run
these CSV files through what's called an "importer", that converts this CSV data into
data structures that Beancount understands. I then append all those extracted entries
into my current .beancount</code> file (which is the main file containing all</strong> my
financial transactions in plain text). I then go through each entry one by one and make
sure it's balanced (in double-entry bookkeeping, all the postings in a transaction must
sum to zero, and not all postings/transactions that an importer outputs are balanced).
Some of that balancing is manual and some of it is automated (e.g. the importer code can
look at the transaction's description and decide which account it should go into, and
balance automatically). This last part (balancing) is where most of those 30-45 minutes
go.</p>
Whenever a new year starts, I move all the transactions from the past year into a
<year>.beancount</code> file and add an include <year.beancount></code> in the active
main.beancount</code>file, mostly to avoid the main file from becoming too long. Not that it
would be an issue for Beancount, but just for the sake of readability.</p>
With such a workflow, all my financial transactions from the beginning of time are
contained in a few plain text files in one directory on my laptop.</p>
Building Beancount Importers for German Banks</h2>
Beancount only provides the foundations for working with money, but it has no knowledge
of what your bank statements look like. This is where the concept of an importer comes
in. An importer is a (Python) class that takes in a bank statement of sorts (e.g. a CSV
export of your transactions) and converts them into something that Beancount can work
with.</p>
I live in Germany and my bank accounts are with German banks, so I had to write a few
importers for a few different banks, specifically beancount-dkb</a>, beancount-ing</a>,
beancount-n26</a>, and beancount-commerzbank</a>. I closed out my Commerzbank account a
while ago, so I don't maintain that integration anymore, but the first three
libraries are actively maintained (and used)!</p>
From User to Author</h2>
My start with Beancount was a bit bumpy. The documentation is very comprehensive but
as a newcomer, I found it tricky to get a grasp on the overall workflow. It took me some
trial and error to figure things out and have that "aha" moment.</p>
I figured that if I found it tricky, maybe it's tricky for others as well. So I wrote
a short book to help newcomers get up and running with Beancount easily. If you're
interested, here's a link: https://personalfinancespython.com</a>.</p>
The feedback on the book has been super positive. It got mentioned on Beancount's
external contributions</a> page, and the reader reviews</a> have all been very encouraging!</p>
Closing Thoughts</h2>
Having all my finances in a bunch of plain text files tracked in a git repository feels
invaluable to me. And hitting the 10 years mark on that almost feels like a milestone.</p>
Perhaps the nicest bit about all this is that this data is sitting on my own
machine</strong>, not in some data center somewhere else. All of it is in plain text files that
I can open up in my editor, and analyze using the tools that the Beancount ecosystem
gives me. All of it will outlive any app or service, and that, I feel, is why plaintext
accounting is so powerful.</p>


HTMX and Django make status polling trivial
2025-11-23T00:00:00+00:00
I work on a few projects in my spare time, and in one of them, I recently had to
implement a simple version of status polling. The main application is built using Django
and I don't have a lot of tolerance for working with and maintaining JS build pipelines,
so I looked into htmx</a> to solve the problem.</p>
And I'm so</em> thankful that I did that. Here's a quick writeup.</p>
Background</h2>
The project in reference is txtcv</a>, a plaintext (JSON resume based) CV building and
hosting platform for software developers.</p>
One of the project's features is cover letter generation. Given the contents of your CV
and a job description, txtcv is able to generate a cover letter for you. This is
implemented as a background job. Basically, the user clicks on a "generate cover
letter" button that sends a POST request to the server to create an empty CoverLetter</code>
object in the database, that starts a task in the background (using a worker process)
that calls out an LLM to populate the contents of the cover letter. As a result of this
workflow, the status of the cover letter can go from "pending" to "running", to either
"successful" or "failed".</p>
Problem</h2>
Given this workflow, it's helpful to let the user know the current status of the cover
letter, so they don't need to refresh all the time to know where things stand.</p>
In a typical backend/frontend split, this sort of thing would be implemented using an
API returning a JSON representation of the CoverLetter</code> object. The frontend would be
set to poll the backend until the status is one of the final values (either successful
or failed).</p>
The problem is that neither do I have (or want) an API for this purpose, nor do I use a
frontend framework in this project. All the pages in the application are rendered 100%
server-side using Django, and I'd like to keep things that way, as long as it's
possible.</p>
htmx + Django</h2>
I had often read about htmx – a library to add dynamic behavior to webpages without any
fuss – but never really had a chance to use it in a project. This problem felt like a
perfect fit for htmx.</p>
So, I installed django-htmx</a> in my project, added a {% htmx_script %}</code> to the base
Django template, and got to work.</p>
Status polling with htmx and Django</h2>
It turns out that the workflow for implementing status polling using htmx and Django is
super simple. While there are multiple ways to do it, this section describes just one.</p>
I added a Django view function in the backend that renders a cover letter template
partial (all server side), given a cover letter ID. The Python code looks roughly as
follows:</p>
from</span> django.shortcuts</span> import</span> get_object_or_404, render</span></span>
</span>
from</span> app.models</span> import</span> CoverLetter</span></span>
</span>
def</span> render_cover_letter</span>(request, id:</span> UUID</span>):</span></span>
    cover_letter</span> =</span> get_object_or_404(CoverLetter,</span> id</span>=</span>id</span>)</span></span>
    </span></span>
    return</span> render(</span></span>
        request,</span> "cover-letter.html"</span>, {</span>"cover_letter"</span>: cover_letter}</span></span>
    )</span></span></code></pre>
Fairly simple. We get a cover letter ID from the URL, render the Django/HTML template,
and send it back as a response to the browser.</p>
The cover-letter.html</code> template is where the real magic happens, so let's have a look
at that.</p>
<</span>div</span></span>
    {% if not cover_letter.has_final_status %}</span></span>
    hx-trigger</span>=</span>"every 2s"</span></span>
    hx-get</span>=</span>"{% url 'cover-letter' id=cover_letter.id %}"</span></span>
    hx-target</span>=</span>"this"</span></span>
    hx-swap</span>=</span>"outerHTML"</span></span>
    {% endif %}</span></span>
    ></span></span>
    <</span>div</span> class</span>=</span>"flex items-start space-x-4 p-6 bg-base-100 rounded-lg border border-base-300"</span>></span></span>
        ...</span></span>
    </</span>div</span>></span></span>
</</span>div</span>></span></span></code></pre>
The code inside the outermost <div></code> is focused on rendering the cover letter card UI,
which is not important for the purpose of this blog post. The real magic happens through
the conditional hx-</code> attributes added to that outer <div></code>. Basically, we're telling
the browser that if the status of the cover letter is not one of the final values, then
fetch the /cover-letter/<id></code> URL every 2 seconds and replace the <div></code> with the
response from the server. And if the cover letter has a successful or failed status,
then all those hx-</code> attributes are not even rendered in the Django template. All of
this logic is controlled server side, without a single line of frontend JavaScript.</p>
Another neat thing is that whatever the "final" status of the cover letter is, the
frontend doesn't need to know or care. At the moment it's abstracted into a @property</code>
on the CoverLetter</code> Django model, so external code can just use that property instead
of having to know what exactly those final status values are.</p>
Conclusion</h2>
And there we have it, status polling made trivial through htmx and Django, without a
single line of frontend JS code or build pipelines.</p>
No API endpoints to maintain, no frontend framework to set up, no JavaScript to write.
Just a Django view that renders a template, a few htmx attributes, and the browser
handles the rest. What could have been a complex feature turned out to be super
straightforward, thanks to htmx.</p>


Building a CV builder
2025-11-01T00:00:00+00:00
A few weeks ago, I started playing around with the idea of developing a CV builder,
specifically for software developers. It came out of a personal frustration that I've
had for a long time now, which is, the options available to me to write and maintain my
CV are honestly not very good.</p>
For the longest time, I kept my CV in LaTeX, which is something I really don't want to
touch anymore. First of all, the language itself is not very intuitive. Next, installing
gigabytes worth of LaTeX packages locally just to compile a text file to a PDF is not
something I like doing very often. And I haven't even gotten to installing external
packages (such as themes) yet. Overall, LaTeX hasn't been a very pleasant experience, so
I'm ruling it out for the foreseeable future for maintaining my CV.</p>
I then switched over to using a third-party app, which was okayish, except that they
charged me something like $20 per month. I generally don't mind paying for services on
the internet where I feel I'm getting enough value, but in this specific case it felt a
bit too much. Besides, the whole experience felt very non-portable to me. In the end, I
saved all my CV data on someone else's website, making my own CV opaque from myself.</p>
Finally, I switched over to keeping my CV in a Google Doc, which kinda sorta works, but
every single time I want to update it, I inevitably end up adjusting things like
document margins, which is annoying.</p>
So, I'm now working on shaving this yak.</p>
I'm working on txtcv</a> to help folks in tech (including myself) build and maintain their
CVs. Here are the top three highlights from the project:</p>

txtcv is portable, using plaintext files to store CV data. JSON Resume</a> is a pretty
nice and mostly complete schema to represent CVs. txtcv only accepts CVs written in
JSON (YAML support is in the pipeline) that comply with the JSON Resume schema. At
all times, the data remains in your control. Put it in a git repository or edit it
using the web interface – it doesn't really care. You can get the JSON out anytime
you want.</li>
txtcv is cheap. The basic plan is free and it lets you maintain one CV. If you want
more fancy features (e.g. CVs tailored to job description, mark your CV web link as
private, AI-assisted cover-letter drafting etc.), the subscription plans start at $25
per year at the time of this writing, which comes down to roughly $2 per month.
You're probably paying a lot more for coffee every day</em>.</li>
txtcv has first-class developer integration. There is a CLI</a> you can install locally
or in Github Actions (more Git providers are in the pipeline) to validate</a> and
publish</a> your CV from anywhere you want.</li>
</ol>
If this piques your interest, head over to https://txtcv.com</a>, sign in using your
Github account, and set up your first CV. Install the CLI via Homebrew if you'd rather
drive the whole thing from the command line. And of course, feel free to follow the
txtcv blog</a> and @txtcv.com</a> Bluesky account for the latest updates.</p>
The project is still very early, so every bit of feedback helps shape what it becomes!</p>


Notes from my first marathon
2025-10-13T00:00:00+00:00
I ran my first marathon in Munich</a> yesterday! The experience was incredible and the
crowd was overflowing with energy. It honestly felt like one giant party across the
city. I thought I'll reflect a little bit on my first time running 42km, covering the
three biggest learnings and my main takeaway, as long as everything is still fresh in my
mind.</p>
A good training plan is critical.</h3>
All in all, I trained for about 3 months with a local running coach who
prepared a training plan for me based on my current (at that time) running status.</p>
The plan was in the form of an Excel sheet that told me exactly what kind of a run
(distance, intervals, etc.) I needed to do on any given day, including rest days,
strength training days, and so on. Looking back, I feel that having this plan was
critical. It was still possible for me to talk to the coach to move things around if
there was a need, but generally, I really liked having this predictability. Plus, it was
really motivating to do the training for the day and put a checkmark in the excel sheet
when I was done!</p>
And no, it really wasn't expensive to get such a plan made. Gym memberships cost a lot
more.</p>
Listen to your body.</h3>
Before I started training, there were a few months where I couldn't find any time at all
to go running. This meant that I went from no running to running ~15km within just a few
days.</p>
This was incredibly stressful on my knee and I ended up developing a bit of a runner's
knee situation. Thankfully, I noticed this very soon and went for a few physiotherapy
sessions, which made a big difference very quickly. Things would've been much worse
otherwise!</p>
Similarly, about 2 weeks before the marathon day, I caught a cold and had to skip a few
training runs, one of them being a ~30km. I had already done a 30km run the weekend
before that one, but this would've been the second time and I was really looking forward
to running a 30km again, but couldn't because of the sickness. The only choice I had was
to sit it out and let the body recover.</p>
If you're in the middle of your training and your body is asking you to do something,
listen!</p>
Mental resilience is just as important.</h3>
Training for and running a marathon requires not just physical strength but also strong
mental resilience.</p>
During the training phase, there were days when going for a run felt like the last thing
I wanted to do, whether due to bad weather or simply not being in the mood. Even during
the marathon (especially between kilometers 24 and 30), almost every few minutes I could
hear my mind tell me to give up and jump out of the race, or just walk the remaining
distance because that's the easier thing to do.</p>
The mind plays tricks, but all of that is normal. Part of the training is to ignore all
those voices, and just keep going.</p>

My biggest takeaway from training for and running the marathon is learning just how
resilient and malleable the human body is. Every time you push your limits, the body
initially feels exhausted. However, if you do it again, it quickly becomes the new
normal.</p>
Being able to do a marathon has been one of the hardest but also one of the
most rewarding experiences I've had so far. And I can't wait to recover and sign up for
my next one!</p>


Good Lazy, Bad Lazy: AI usage reflections
2025-09-27T00:00:00+00:00
It's been a while since I integrated AI agents into my programming workflow. At work I
mainly use Claude Code. For private projects I'm a cheapskate, which means I use
whatever is the cheapest (which at the time of this writing happens to be Gemini, given
their very generous free plan).</p>
After about a year of using them almost on a daily basis, I've noticed my AI usage falls
into two (generally) distinct camps.</p>
Camp 1: Productivity Gains</h3>
The first camp is what I'd call productivity gains. This is when I offload the type of
work that's necessary but is either rote work or is somewhat at the edge of my
domain knowledge.</p>
For instance, if I'm working on a fullstack feature and want to make the UI look good
(building UIs not being my core skill), I may use an AI agent to take a first pass
at the implementation. This saves me a few rounds of Google searches, copy/pasting
suggestions from StackOverflow, etc.</p>
Although implementing good looking UIs may not be my core skill, I'm still fairly
confident at distinguishing a good implementation from a bad one, which means I have a
very good idea of what I'm looking for. Using AI agents to help me with such problems
saves me time and increases the scope of what I'm capable of building, in turn making me
more productive.</p>
Camp 2: Good Lazy</h3>
The second (more interesting) camp is when I'm being plain lazy, but of the good kind!</p>
A good example here is creating pull requests on Github. At work, I wrote a custom
slash command</a> for Claude Code to create PRs on Github. Instructions for this command
ask Claude to take a look at the diff between the main development branch and the
current branch, and create a PR on Github using the gh</code> CLI, assigning it a title and
description based on the git diff and formatting the description based on the PR
template checked in to the git repository.</p>
Of course, I could easily do all those things myself, and the PR's description would be
a bit more focused (LLMs tend to blabber a lot!), having a custom command for this saves
me time. I'm being lazy, but I'd argue that this is the good lazy.</p>

At the end of the day, I feel that when using AI agents, it's very important to know the
difference between genuine productivity gains and shortcuts to complacency. While the
two of them are not mutually exclusive, comfort that leads to complacency isn't
something desirable in the long run. Hopefully this quick reflection of mine encourages
you to be mindful of your own workflows with AI agents!</p>


Starting fatherhood
2025-02-21T00:00:00+00:00
Earlier this week, my wife and I welcomed our tiny little daughter in this world. As
parents, we're very happy and completely smitten by this new addition to our lives.</p>
I'm barely a week into fatherhood, so there's a lot to learn. I didn't fully grasp what
people meant when they said that becoming a parent changes you at a fundamental level.
Now I think I'm beginning to understand. Taking care of a newborn is a responsibility
unlike anything else I've experienced. It's challenging, has a steep learning curve, and
the stakes are high. Yet, at the end of the day, when you see that tiny human look at
you and smile, nothing else matters and you feel like you'd gladly do it all over again.</p>


Trying out Zed after more than a decade of Vim/Neovim
2025-01-25T00:00:00+00:00
I'm currently drafting this blog post in Zed</a>. After using Vim/Neovim for more than 15
years, I recently decided to try out something new. I don't know if this little
experiment will work, or if I'll run back to my trusty Neovim, but hey, what I do know
is that I'd like to find out.</p>
Why try something else?</h2>
Why though? If you've used a specific tool for that long, why (try to) switch to
something else anyway? Everyone has their own reasons. I have two.</p>
1. The desire to use something that just works</h3>
Lately, I've been drifting towards things that "just work".</p>
I love (Neo)Vim. Like I wrote earlier, it's been my primary editor for the past 15
years. What I don't love is all the configuration that goes into it before I can use it
to start writing code.</p>
When I first started using Vim (in 2009!), my configuration was a tiny .vimrc</code> file
that was handed over to me by my internship supervisor at that time. I had no idea what
it did, but it worked. Over time, as I felt the need to customize things, my .vimrc</code>
started accumulating increasingly more code copied from StackOverflow that I didn't
understand.</p>
At some point Neovim came out, and with it, the ability to configure things using Lua.
This was a huge step up, because I could at least understand what I was configuring.</p>
Either way, it's still configuration. What changed was just how</em> that configuration was
done. The combination of configuring the base editor and installing a set of plugins
(and making sure that they play nice with each other) isn't really something that I
would like to spend my time on, going forward.</p>
One recent example that highlights this problem: my workflow consists of switching
back and forth between a terminal window and an editor window. At any time, I have
multiple projects open in both. On macOS, it's been difficult to find a solution that
"just works". The most popular Neovim GUI clients on macOS include Vimr</a> and Neovide</a>,
which are both excellent projects. Neovide though, does not support multiple windows</a>,
which is integral to my workflow and hence requires me to implement workarounds</a>, which
actually broke last week after I ran :PackerUpdate</code>. And while Vimr does support
multiple windows, it is not able to render icons in nvim-tree</a>, which is less than
ideal. 🤷🏻‍♂️</p>
2. Deeper and native LLM integration</h3>
LLMs are happening, whether we like it or not. And no, the fact that they aren't 100%
correct all the time is not a reason to discard them entirely. I understand why some
people don't want to use them. And I respect that. I personally find them useful and
would like to integrate them more into my daily workflow.</p>
The Neovim ecosystem has a bunch of plugins for using LLMs. But as I wrote in the
previous section, I'd like to avoid plugins when possible. Installing plugins means
updating them, which inevitably breaks things.</p>
One example where I find LLMs useful when coding is handling boilerplate stuff. In my
current Neovim setup, I have the the Codeium extension</a> installed, which offers code
suggestions that are like an autocomplete on steroids. Often, I can write a function
name and what parameters it's expecting, and the LLM writes out the function for me.
This is particularly helpful when writing small functions or test cases.</p>
I'd like to be able to do more of such things in my editor. I'm not entirely sold on the
idea of agentic editors, as I'd like to keep at least some</em> agency over the code that's
going in, but that's a different topic.</p>
Another example of where I find LLMs useful: I'm not a native English speaker and often
struggle to find the right sentence framing that is both concise and catchy. LLMs are
excellent at this! I can scribble my initial thoughts, throw it at an LLM and ask it to
reframe the whole thing, which is great!</p>
Generally, it feels like the way we're writing code is changing with the introduction of
this extremely powerful tool, and I'd like to move with the times.</p>
Why Zed?</h2>
So yeah, those are the two reasons why I've been looking to try out something else
lately. What made me go for Zed?</p>
1. Vim mode</h3>
Yep, that's literally the first reason. After using Vim for 15 years, my fingers have
built up enough muscle memory that not using Vim keybindings when writing code is
downright impossible.</p>
Zed's Vim mode</a> is surprisingly solid! So far, I feel right at home. Almost all Vim
keybindings that I'm used to work just as expected. The one or two bindings that don't
work, are something that I can make my peace with and retrain my fingers on. Everything
else works excellent. It also looks like they're doubling down on their Vim mode</a>
support in 2025, which is a great sign!</p>
2. It just works</h3>
When you fire up Zed, the editor is fully functional without having to write a single
line of configuration. That's awesome. The editor may prompt you every now and then to
install support for specific languages. But that's usually just one button click.</p>
3. JSON</del> Less configuration</h3>
Update: this one started a bit of controversy on internet water coolers so I'll
elaborate a little more.</em> 🙃</p>
Lua is great, but one JSON file is even better. Zed uses JSON as the configuration
syntax and so far it feels a lot simpler than what I've worked with so far.</p>
The underlying sentiment here is that I'm looking to have less</strong> configuration. I
don't really care whether the format is JSON, or YAML, or TOML, or whatever else. I'm
actively trying to do less dynamic things in my configuration, so I'm beginning to
appreciate the fact that the text editor gives me less buttons to play with.</p>
4. Configuration autocomplete</h3>
I had no idea I needed this until I had it!</p>
</p>
While editing the Zed configuration file, I found it really helpful how the editor
suggests configuration keys and their potential values. This feature felt really
thoughtful when I first saw it and I could imagine that it goes a long way in getting
new users up to speed quicker.</p>
5. Native LLM integration</h3>
I know some people have been turned off by this, but I actually like this feature.</p>
Zed has a feature called Assistant, which is a tool to, well, assist you, using a large
language model of your choice. A few popular LLM providers are supported. Once set up,
you can open up the Assistant panel to work together with the language model you've
chosen. For instance, it's pretty easy to type in questions in the panel window and then
paste text from your open file buffers into it to give the LLM more context. There's a
lot more to this feature that I haven't used yet, but I'll probably get to it the more I
use it.</p>
Overall, the integration feels very native and very useful.</p>
6. Fast!</h3>
Last but definitely not the least is speed! Zed is very</em> fast. Everything feels very
snappy, and it's clear that the team has put a lot of effort into making everything
fast.</p>
Conclusion</h2>
It hasn't been that</em> long since I really started using Zed instead of Neovim as my
daily driver. And so far the experience has been quite nice.</p>
Like I wrote earlier, this is an experiment. We'll see how things turn out. That being
said though, the first impression looks very</strong> good.</p>


Building Cython (or C) extensions using uv
2025-01-11T00:00:00+00:00
Building and maintaining Python libraries with extension modules can be somewhat tricky.
I maintain one such library called streaming-form-data</a>, that provides a streaming
parser for multipart/form-data input and has all the performance-critical code written
using Cython.</p>
In this post I'll share my experiences transitioning this library from Poetry to uv. If
you also maintain such a library or are just plain curious about how uv can work
together with Cython, you may find this post useful.</p>
Why transition to uv?</h3>
For years, I've been using Poetry for streaming-form-data</code>, together with a custom
build.py</code> script for handling the C extension. While this setup is not explicitly
documented / supported by Poetry, it has worked really well so far.</p>
Last week though, the release of Poetry 2.0.0 introduced a bug that broke my build
process, and introduced installation errors for some users of the library. After
spending some time debugging the issue, I realized that this issue would be best fixed
upstream (at Poetry's end). At the same time I've been looking into uv</a> with curiosity
for a few months now, as I'm sure most of the members in the Python community have been.
I wasn't actively looking to move away from Poetry, but this seemed like a good
opportunity to try uv out!</p>
The move went pretty well! I published version 1.19.1 of the package yesterday which was
the first version built using uv!</p>
uv and build backends</h3>
uv is the latest tool in the Python ecosystem that has taken almost the entire community
by storm. It's extremely</strong> fast. It consolidates multiple tools into one. It can
manage the installation of multiple Python distributions for you. And did I mention that
it's fast?</p>
uv also allows building and publishing packages to PyPI, delegating the actual package
building to something called a "build backend". Popular build backend choices at the
time of this writing include hatchling</code> (ideal for pure Python projects), setuptools</code>
(versatile and battle-tested choice that supports C extensions), flit</code> (focused on
simplicity for pure Python projects), and a few others.</p>
setuptools</code> remains a solid choice for projects that use Cython or include a pure C
extension. Despite misconceptions about its status (no, it's not deprecated),
setuptools</code> continues to evolve and is actively maintained by the PyPA.</p>
It's also what I ended up going with for streaming-form-data</code>.</p>
Cython</h3>
Cython is a superset of Python that gives you the ability to compile your Python code
into optimized C code, which can then be included as a normal C extension into your
library. There are several ways of distributing Cython code in libraries. I personally
use the following approach:</p>

Compile the .pyx</code> source code into .c</code> using the cython</code> CLI.</li>
Include the generated .c</code> file into the source distribution, so that end users don't
need to have Cython installed.</li>
</ol>
I've been using this approach for quite a few years now and haven't had any reasons to
complain so far.</p>
pyproject.toml configuration</h3>
Here's how I configured all the above in the project's pyproject.toml</code>:</p>
[</span>build-system</span>]</span></span>
requires = [</span>"setuptools"</span>]</span></span>
build-backend =</span> "setuptools.build_meta"</span></span>
</span>
[</span>tool</span>.</span>setuptools</span>]</span></span>
package-dir = {"" =</span> "src"</span>}</span></span>
ext-modules = [</span></span>
    {name =</span> "streaming_form_data._parser"</span>, sources = [</span>"src/streaming_form_data/_parser.c"</span>]}</span></span>
]</span></span></code></pre>
The build-system</code> section defines what build backend to use. package-dir</code> in
tool.setuptools</code> maps the package directory to src</code>, aligning with modern Python
packaging recommendations. And finally the ext-modules</code> list defines the C extension to
be built, using the .c</code> file generated by Cython.</p>
Conclusion</h3>
In my case, the transition to uv has worked quite well!</p>
One standout feature is its remarkable speed. For instance, in my Github Actions
pipeline, the step for installing the package dependencies now takes only 7 seconds,
down from 14 earlier. Resolving packages locally is almost instantaneous, and the entire
experience is incredibly snappy. What I appreciate most, though, is the simplicity  of
managing just one tool (uv) instead of juggling both pyenv and Poetry.</p>
Beyond its performance, uv is rapidly gaining traction in the Python community. While I
typically wait a few months before switching to the shiny new object, this time, the
switch has been worth it. If you're considering making the move, uv is a great choice!</p>


Moving from Beancount 2.x to 3.x
2024-07-10T00:00:00+00:00
A few days ago, Beancount officially switched</a> from 2.x to 3.x. If I'm being honest, I
was a bit afraid of this release, mostly because it was supposed to be a major rewrite.
I'm generally not a huge fan of rewrites, and I don't think I'm the only one on the
Internet who thinks that way.</p>
Luckily, the update seems to have been less of an event as I anticipated. If you're
planning to move from Beancount 2.x to 3.x, the update should be mostly</em> uneventful.
Nevertheless, there are a few important changes that may affect your workflow. The rest
of this blog post will summarize the ones that I've noticed so far.</p>
1. External data import workflow</h2>
Update (May 2025)</strong>: The original version of this blog post that I published in July
2024 described only the pyproject.toml-based approach of importing external data. Since
then, I've discovered another (somewhat more official) way of importing external data,
using beangulp</a>. Although the official documentation does not seem to reference it yet,
beangulp provides a few examples that use the Ingest</code> class to set up a workflow that
is not so different from the older config.py</code> based workflow. For completeness, I'll
describe both the approaches here.</em></p>
The first thing I noticed is that the workflow for importing external data looks
different.</p>
With 2.x, you had to maintain a config.py</code> file in your project root where you would
import all your Importer classes and put them inside a CONFIG</code> variable.</p>
from</span> first_bank_importer</span> import</span> FirstImporter</span></span>
from</span> second_bank_importer</span> import</span> SecondImporter</span></span>
</span>
CONFIG</span> =</span> [FirstImporter(), SecondImporter()]</span></span></code></pre>
The Beancount commands like bean-extract</code> and bean-identify</code> would then pick this file
and run through the list of specified importers to find the matching one.</p>
In 3.x, this workflow is gone. Commands like bean-extract</code> and bean-identify</code> have
been removed from the beancount</code> package. Instead, the project has decided to go for a
script-based workflow instead (source</a>). It appears that you don't need to maintain a
config.py</code> anymore, and your importers need to provide a command-line entry point that
can do those tasks (eg. identifying whether a given file matches an importer or
extracting a list of transactions out of a file).</p>
If you maintain custom importers and distribute them through PyPI, the one immediate
downside of this approach is how those importers are initialized. With the config.py</code>
approach, your users could instantiate importers just as any Python code, because it was
literally just Python code. Here's an example of how that used to look like:</p>
from</span> my_importer</span> import</span> MyImporter</span></span>
</span>
CONFIG</span> =</span> [</span></span>
    MyImporter(</span></span>
        name</span>=</span>"Assets:MyBank:Checking"</span>,</span></span>
        currency</span>=</span>"EUR"</span>,</span></span>
        patterns</span>=</span>{</span>"ALDI"</span>:</span> "Expenses:Supermarket:ALDI"</span>},</span></span>
    )</span></span>
]</span></span></code></pre>
In the absence of config.py</code>, you'll need to find a different way of doing this.</p>
1. Using beangulp.Ingest</code></h3>
beangulp</code> provides an Ingest</code> class that accepts a list of importer objects,
ultimately giving you a workflow similar to what was available using config.py</code>. The
idea is to define an import.py</code> script (the script's name can be anything) in your
project root that contains something like the following:</p>
from</span> my_importer</span> import</span> MyImporter</span></span>
from</span> beangulp</span> import</span> Ingest</span></span>
</span>
importers</span> =</span> (</span></span>
    MyImporter(</span></span>
        name</span>=</span>"Assets:MyBank:Checking"</span>,</span></span>
        currency</span>=</span>"EUR"</span>,</span></span>
        patterns</span>=</span>{</span>"ALDI"</span>:</span> "Expenses:Supermarket:ALDI"</span>},</span></span>
    )</span></span>
)</span></span>
</span>
if</span> __name__</span> ==</span> "__main__"</span>:</span></span>
    ingest</span> =</span> Ingest(importers)</span></span>
    ingest()</span></span></code></pre>
Running this script should produce the following output on the console:</p>
Usage: import.py [OPTIONS] COMMAND [ARGS]...</span></span>
</span>
  Import data from and file away documents from financial institutions.</span></span>
</span>
Options:</span></span>
  --version  Show the version and exit.</span></span>
  --help     Show this message and exit.</span></span>
</span>
Commands:</span></span>
  archive   Archive documents.</span></span>
  extract   Extract transactions from documents.</span></span>
  identify  Identify files for import.</span></span></code></pre>
As you can see, the script now supports archive</code>, extract</code>, and identify</code>
subcommands, that replace bean-file</code>, bean-extract</code>, and bean-identify</code>,
respectively.</p>
2. Defining importer-specific CLIs</h3>
If you don't want to use beangulp.Ingest</code>, another approach is put the importer's
initialization parameters inside pyproject.toml</code>. The pyproject.toml</code> file</a> is
a relatively new Python standard that is used by a few other tools in the Python
ecosystem to place their configuration bits (eg. inside a tool.X</code> section), amongst
other information. The reasoning here is that if you're using Beancount, your finances
project is basically a Python project, that may as well contain a pyproject.toml</code>
file, which in turn means putting importer configuration in there isn't all that bad.</p>
As an example, if you maintain a beancount-mybank</code> package distributed through PyPI,
you could ask your users to put the following snippet inside their pyproject.toml</code>:</p>
[tool.beancount-mybank.ec]</span></span>
name = "Assets:MyBank:Checking"</span></span>
currency = "EUR"</span></span>
patterns = [</span></span>
    ["ALDI", "Expenses:Supermarket:ALDI"]</span></span>
]</span></span></code></pre>
... and add a beancount-mybank-ec</code> CLI command as part of your beancount-mynank</code>
package. This CLI command could read the configuration inside pyproject.toml</code>,
initialize the importer, and then call out identify</code> or extract</code>.</p>
Of course, this is just one way of approaching things. I'm sure there are more ways to
approach this problem.</p>
Although between the two approaches mentioned above, I would recommend going with the
beangulp</code> approach.</p>
2. Fava compatibility</h2>
Update (May 2025)</strong>: Fava does support Beancount 3.x</a>!</em></p>
This one is unfortunate. It seems like Fava isn't compatible with Beancount 3.x, yet.
There are a couple of Github issues (eg. #1824</a> and #1831</a>) that are open at the time
of this writing. So far I haven't seen much activity on those issues, so I'm not sure
what the current state is.</del></p>
If you're upgrading to Beancount 3.x, be aware that you'll lose out on Fava.</del></p>
3. Adjusting Importers</h2>
The next thing I noticed is that the importer class definitions need to be adjusted
because of API changes.</p>
The beancount.ingest</code> package is gone and has been replaced with the beangulp</a> module
which contains the new importer base (abstract) class and a few other utilities to work
with external data. The changes are not many and are not very big either. Here's a quick
list:</p>
3.1. Updated base importer class</h3>
The first change is, of course, that beancount.ingest</code> is gone. Well, not completely
gone, but at least gone from beancount</code>. This functionality has now been extracted to
the beangulp</code> module.</p>
In Beancount 2.x the importers were inheriting from
beancount.ingest.importer.ImporterProtocol</code>. In 3.x, they should now inherit from
beangulp.importer.Importer</code>.</p>
from</span> beangulp.importer</span> import</span> Importer</span></span>
</span>
class</span> MyImporter</span>(</span>Importer</span>):</span></span>
    ...</span></span></code></pre>3.2. Updated file pointer type in method signatures</h3>
The second change is in the method signatures of methods like extract</code> or identify</code>.
In beancount.ingest.importer.ImporterProtocol</code>, these methods used to accept a
file-like object (cache._FileMemo</code>) as a parameter.</p>
def</span> extract</span>(self, file: cache._FileMemo):</span></span>
    pass</span></span></code></pre>
This file-like object has been replaced with a string filepath, which is a bit more
straightforward to work with.</p>
def</span> extract</span>(self, filepath:</span> str</span>):</span></span>
    pass</span></span></code></pre>3.3. Importers don't have self.FLAG</code> anymore</h3>
This is a minor one, but I think it might be worth mentioning. If your importers are
using self.FLAG</code> in any of the methods, it won't work anymore. You'll need to replace
it with one of the flags defined in beancount.core.flags</code>.</p>
3.4. Method names</h3>
And the final change I noticed is how the importer method names have changed.</p>
beangulp.importer.Importer</code> defines a new interface that importers should implement.
The identify</code> and extract</code> methods have been kept. Other than that, the
file_account</code>, file_date</code>, and file_name</code> methods have been renamed to remove the
file_</code> prefix. So file_account</code> has become account, file_date</code> has become date</code>, and
file_name</code> has become filename</code>.</p>
The class is defined here</a>, in case you'd like to see the complete interface yourself.</p>

And that's pretty much it. After making the above changes to your importers and your
workflow, your setup should</em> be compatible with Beancount 3.x. As I mentioned earlier,
the list of changes is not very big. And the changes themselves aren't that big either.
However, I couldn't find a migration document that would guide me through the process,
so I thought about writing a quick one up myself.</p>
I hope this blog post was helpful. If you have any questions (or noticed something in
this blog post that is not correct), please feel free to reach out to me on Bluesky</a>!</p>
(Oh and btw, if you'd like to learn more about how you can track your personal finances
using Python and Beancount and don't mind sticking with Beancount 2.x for now, I wrote
a book</a> on the topic. 🙃)</p>


Handing over the reins of GeniePy
2024-06-25T00:00:00+00:00
A few days ago, I finished the process of handing over the maintenance and further
development of GeniePy</a> to a new owner. I've worked on the project for around 3 years
and after all this time, it was finally time to sell. In this blog post I'll briefly
describe the process.</p>
Starlette beginnings</h2>
I started building GeniePy sometime back in 2021. At that time I had a lot of SaaS
business ideas but the first step towards building any of those was to build the basic
SaaS functions first - user registration, login, payments, password resets, the list
goes on.</p>
So I built GeniePy mostly for myself. The idea of building and selling a SaaS
boilerplate in Python wasn't entirely new. Indeed, a few SaaS boilerplates in Python
already existed on the market. What was different was the technology stack. Every
engineer has a slightly different taste in terms of tech stack, and mine at that time
involved Starlette</a>. So I wrote the first version of GeniePy on top of Starlette.</p>
Over time, I realized that even though Starlette was (and continues to be) an excellent
framework, it wasn't very well known in the startup world. FastAPI</a>, which is an API
framework built on top of Starlette, was (and is) extremely popular, but unfortunately
Starlette wasn't. This was a problem. Making online sales is already pretty difficult.
And if the product has an inherent quality that can potentially act as a blocker in
making sales, that's not a very good situation to be in.</p>
The rewrite in Reflex</h2>
Around mid 2022, the Python ecosystem saw the launch of a framework called Reflex</a>.
Reflex came with the promise that you could code your entire</em> application (backend and
frontend) in Python. All you had to do was stick to the data structures and primitives
that Reflex offered. And at deployment time, your code would be compiled into a FastAPI
backend and a Next.js frontend, all taken care of by Reflex. You could deploy the
FastAPI server as you would deploy any other backend API. And the Next.js frontend could
be deployed as a static site (or as any other Node application).</p>
This sounded very exciting to me. After the realization that I would have to move away
from Starlette, I was looking for other frameworks to rewrite the boilerplate in. Reflex
seemed like an excellent fit. At that time I was between jobs, so I decided to just go
for it. It took me about 2-3 weeks to finish the rewrite (in whatever time I had left
over from job interviews). I was even able to find time to take part in the Reflex
community. I joined their Discord, managed to submit a few PRs</a> to the Reflex
framework, and even helped them test out the beta for their hosting service</a>. The
Reflex community was very welcoming and friendly to this newcomer.</p>
After the rewrite was complete, I launched the updated version</a>. That launch felt very
good. It brought a lot of new energy to the project. And it felt like the updated
version would be very useful to Python developers looking to build and ship SaaS
applications quickly. I wrote a few blog posts, posted the links to a few online water
coolers, and managed to get some more attention to the project, some of which turned
into customers.</p>
Time</h2>
After the rewrite to Reflex and the overall positive response, I started to have users
ask me for more features.</p>
I managed to ship quite a few of those. But there were a few others which were excellent
feature suggestions but a bit too heavy of a lift to be done within the spare time I
actually had. At around the same time, a few changes happened in my personal life which
resulted in me having even less time (and energy) available to devote to side projects.
GeniePy was still active and I was shipping security updates, but new feature
development had effectively paused.</p>
This was the time that I realized that it might be time to move on. I had been working
on the project for close to three years. If I wasn't able to find the time and energy to
work on it, I wanted to find someone else who did.</p>
So, I started looking for potential individuals (or companies) willing to take the
project on and invest more time and resources into it.</p>
Luckily, with the rise of the Indie Hacker movement, the process of selling businesses
online has become a lot more convenient than it used to be. I posted the project on a
few micro-SaaS acquisition marketplaces and received quite a few leads.</p>
Buy // Sell Startups</h2>
A few weeks after posting the project on micro-SaaS marketplaces, I received an email
through Buy // Sell Startups</a> from someone interested in acquiring GeniePy. This was
from Hannes Lehmann running the companies sistemica</a> and devtech.pro</a>.</p>
We exchanged a few emails and had an initial chat on Google Hangouts where we first got
to know each other, and then spoke about the project. We talked about the technology
stack, the revenue GeniePy was making, the future development ideas I had, Hannes's
plans for it, and so on.</p>
The first call went really well. I had the feeling that Hannes really saw the potential
in the project and was willing to invest the time and energy into improving it further.</p>
After the initial chat, we scheduled a few more follow-up calls, where I did codebase
walkthroughs, explained the different workflows I had set up to publish releases,
basically everything that went behind the scenes. After those follow-up calls, we let
everything settle in our brains for a few days, allowing both of us to process all that
new information.</p>
And after that delay, we both felt good about making the transfer. It generally felt
like a very good mutual fit. I personally was very happy that the project could again
get the time and energy it really deserved. We mutually decided upon June 1st as the
date to transfer all the accounts from me to Hannes.</p>
The transfer itself went quite smooth. Coincidentally, Hannes was (is) also based in
Germany, which made the whole thing a lot more convenient because we were both familiar
with conducting business in this country. Honestly, the longest part of the process was
just the DNS delay as a result of moving from one provider to another.</p>
Conclusion</h2>
Overall, I'm super happy about GeniePy having a new owner.</p>
It's been only about a month and GeniePy has already been upgraded to support Reflex
5.0, which was something that the users had started asking for recently. The future
plans documented</a> by the new team also look very much aligned with what the user
feedback over the last few months has been.</p>
I think it's safe to say that the future for GeniePy looks quite bright.</p>


Experiences using Tailwind CSS
2024-05-20T00:00:00+00:00
Tailwind CSS</a> has gained a lot of popularity in the recent years, for many good
reasons. There are a ton of interesting things that it brings to the table.</p>
One thing that I</em> personally found interesting is that a lot of that popularity seems
to come from backend developers, who almost by definition, don't write a lot of code on
the frontend. As someone who also subscribes to that definition (at least to some
extent), I found that bit quite intriguing. Developing code that's visible to the
end-user has always been a bit of an Achilles' heel for me. So the thought of solving
that problem was honestly quite exciting.</p>
So, over the last few evenings and weekends, I rewrote the styling for this site using
Tailwind, mostly to figure out what I can do with it. This post describes my experience
with Tailwind so far.</p>
Background</h2>
Before we begin, it would help to establish a little bit of context. As I wrote earlier,
my core technical competency is backend and infrastructure/devops software development.
I have</em> written frontend code in the past, but it was not where I spent most</em> of my
time. In the past whenever I had to kick-start a project, my first thought was to use
either Bootstrap</a> or Bulma</a>. More often than not, the final result looked as if it was
"engineered" and not "designed" (which was not the fault of the framework but rather a
consequence of my limited design skills). It wouldn't be an exaggeration to say that I
could identify with the following gif.</p>
</p>
Usually when I want to learn something new I like to read a book or a few blog posts to
get up to speed. In this case I chose to dive heads first. I picked my personal website
(the thing you're reading right now) as a playground and decided to rewrite the styling
using Tailwind.</p>
Making Tailwind work with Zola</h2>
This site is built using Zola</a> and deployed on Netlify</a> as a static site. I don't use
a third-party theme, which means that all the HTML code on this site is under my
control.</p>
The first step was to install Tailwind. I ran npm install tailwindcss</code> on the console
and had that out of the way.</p>
The next step was to initialize Tailwind, which is done using npx tailwindcss init</code>.
Running this command writes a tailwind.config.js</code> file in the project root. I had to
adjust the content</code> key to point to the directory where my Zola template files live, so
that the Tailwind CLI can look at what CSS classes the templates are using, and
therefore should be included in the final CSS file served to the browser. I also had to
install the @tailwindcss/typography</code></a> plugin to make sure that the Markdown content
which is rendered to HTML (by Zola) has good typographic defaults. The final
tailwind.config.js</code> looks like this:</p>
/**</span> @type</span> {import('tailwindcss').Config}</span> */</span></span>
</span>
module</span>.</span>exports</span> =</span> {</span></span>
    content: [</span>"./templates/**/*.html"</span>],</span></span>
    theme: {</span></span>
        extend: {},</span></span>
    },</span></span>
    plugins: [</span></span>
        require</span>(</span>'@tailwindcss/typography'</span>),</span></span>
    ],</span></span>
}</span></span></code></pre>
Next, I created a main.css</code> file in the project root with the following contents:</p>
@tailwind</span> base;</span></span>
@tailwind</span> components;</span></span>
@tailwind</span> utilities;</span></span></code></pre>
And then I ran the command tailwindcss -i ./main.css -o ./static/main.css --watch --minify</code>, and let it run in a separate terminal window. This command watches main.css</code>
and all the *.html</code> template files for changes and recompiles (and minifies) the final
CSS file to be included in the source code. The output is written to static/main.css</code>
which is served as the final result using a <link></code> tag.</p>
For simplicity, I checked static/main.css</code> into version control. The alternative would
have been to generate it at build time in Netlify but that would have introduced
additional complexity which I didn't want.</p>
What I like</h2>
Overall, there is a lot</em> to like about Tailwind. My experience has been so positive
that I see myself reaching out for Tailwind before any other CSS framework in the near
future. In this section I'll write about the two things that jumped out the most.</p>
The first was how quick and easy Tailwind made it for me to get things in to a working
state. Adjusting things using CSS has always been an issue for me. Frameworks like
Bootstrap helped, but the moment something was out of scope of the framework (eg.
modifying the appearance when the cursor is hovering over an element), I had to resort
to searching on Google and copy/pasting code without having much of an understanding of
what it was doing.</p>
With Tailwind, I can set things in place really</em> quickly. What helps me is to think of
a page as a collection of components that are mostly independent of each other. This
way, it's possible to define them individually, including thinking about how they should
look like in responsive contexts. The "Flexbox & Grid" utility classes are especially
helpful here.</p>
The second thing that jumped out to me is how easy it is to make things look nice. I
know that "nice" is a subjective word; maybe non-ugly is a better word? In any case, my
experience has been that Tailwind generally makes things look nice by default. And</em>,
there is a vast collection of utility classes that lets you control the look and feel of
essentially everything on the page. It's easy to control spacing using margins and
paddings, control how text looks like, what colors are used (both for the main content
and the background), transitions, and so much more. The number of utility classes is
really enormous and I think I've barely scratched the surface so far. But my impression
so far is that if you can think it, you can build it.</p>
What I don't like</h2>
There's actually not much that I don't like. Regardless, to balance things out, I tried
hard to think about it and was able to come up with two.</p>
The first one is that the HTML code is not as clean as I would like. It's OK for a
personal website like this where I'd like to keep the appearance simple. However, as the
design gets more intricate, the number of CSS classes also explodes. Tailwind authors
suggest using partials or components to solve this problem. I can absolutely agree. I'd
probably even go one step further and say that without partials (or components) it's
quickly going to go out of hand. So if you're using Tailwind, keep it in mind that at
some point, you'll have to create those abstractions.</p>
The second thing that I don't like</del> know what to think of yet is the culture of
copy/paste components that has cropped up in the ecosystem. There are a lot of Tailwind
UI component libraries out there, Tailwind UI</a>, Preline</a>, Flowbite</a> being the most
popular ones. The usage instructions for all these libraries is "here is the HTML code,
copy and paste it in to your project". In one of the side projects I've been using
Tailwind UI components. The components themselves are extremely well done, which also
means that the number of CSS classes is high. That in turn means that if you don't make
some time upfront to go through the code and understand what those classes are doing,
you might end up missing context. Traditionally, when you use a library, the library
puts an abstraction between you and the underlying code (eg. btn</code> instead of rounded bg-indigo-600 px-2 py-1 text-sm font-semibold text-white shadow-sm hover:bg-indigo-500 focus-visible:outline focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-indigo-600</code>). Without any such abstraction, you just have to
invest some time upfront to understand what each of those 13 classes are there for.</p>
Conclusion</h2>
Overall, I've come to really like Tailwind. In my experience, it's an incredibly
powerful tool that allows someone like me (with a technical skillset that has
traditionally not focused on the frontend) to quickly build user-facing websites. This
lets me work on the entire stack instead of being limited to just the backend. It's also
honestly a lot of fun to work with. I love</strong> the instant feedback of applying a
utility class and see the appearance change instantly. I can definitely see myself using
it more and more in the future.</p>


One of the most useful directories on my laptop
2024-03-29T00:00:00+00:00
One of the most useful directories on my laptop is called finances</code>.</p>
Almost every single time I touch it, I'm reminded of its usefulness. Every single year
when we have to file our tax returns, I'm again reminded of how useful it is. What does
it contain, one might ask? It contains a log of all my financial transactions across
different bank accounts (both active and inactive), in a plain-text format, dating all
the way back to the beginning of 2016. At the time of this writing, this is 8 years of
financial activity, in one directory (and it's even version controlled!).</p>
Background</h3>
Before I talk about the cool stuff, let me quickly talk about Beancount.</p>
Beancount</a> is one of the most useful programs (at least to me) in existence. It's an
implementation of the double-entry bookkeeping system in Python. It lets you represent
financial transactions - money moving from one account to another - in an easy-to-read
plain text file. The file is easy to read for both humans and computers. Here's an
example snippet:</p>
2024</span>-</span>04</span>-</span>01 *</span> "Thank you for your purchase."</span></span>
    Assets</span>:MyBank</span>              -</span>10.00</span> EUR</span></span>
    Expenses</span>:Supermarket</span>        10.00</span> EUR</span></span></code></pre>
These three lines make up a transaction that conveys that an amount of €10 moved from
your bank account to your local supermarket's account, perhaps when you went grocery
shopping.</p>
Beancount files are a collection of lots and lots of such transactions. The way
everything works is that you download an export of your financial activity from your
bank (usually a CSV export), run a Python program that converts the CSV to the
.beancount</code> format, and then append everything to your own .beancount</code> file.</p>
Over time, this file starts accumulating more and more of your financial activity and
starts to grow into a documentation of essentially how your money has been moving
between different accounts. This makes answering questions about your financial activity
almost trivial. For instance, if you want to know how much money your side-hustle (let's
call it Apple) made back in 2018, all you have to do is filter for transactions that
deposit money into the Income:Hustles:Apple</code> account (made-up name) and limit the
timeframe to 2018.</p>
The concept is quite simple but it unlocks so many cool use cases. I'll describe one
here.</p>
Efficiently preparing tax returns</h3>
At the end of the year you normally have to file tax returns. At least in the country
where I live, this involves collecting lots and lots of receipts and invoices that you
can send to the tax authorities as proof that your business incurred some expenses, so
you can claim some of that taxed money back.</p>
In my pre-Beancount days, this was the part I never liked. I never knew where to put
receipts. How was I supposed to find and collect them? I had a Documents</code> folder on my
machine. Maybe that's where the receipts should go? Should this folder be organized by
year, or by purpose, or by something else entirely? Also, what if I decide one
organizational scheme one day but then after a few months I find out that that was
completely stupid and I should be using a different organizational scheme? Does this
mean I have to rearrange everything?</p>
Beancount lets me solve this problem very elegantly. You see, Beancount lets me attach
documents to transactions. Again, this is such a simple concept, but it is so powerful.</p>
Let's illustrate using the following transaction.</p>
2024</span>-</span>04</span>-</span>01 *</span> "Thank you for your purchase."</span></span>
    Assets</span>:MyBank</span>              -</span>12.99</span> EUR</span></span>
    Expenses</span>:Netflix</span>            12.99</span> EUR</span></span></code></pre>
Assume that this transaction lives inside finances/example.beancount</code>. If you have a
receipt from Netflix for this payment (and for whatever funky reason you want to show
this as a business</em> expense), you can create a finances/documents/</code> directory and
place the receipt at finances/documents/Expenses/Netflix/2024-04-01.pdf</code>, and that's
it! The Expenses/Netflix/</code> directory tree represents the name of the account and the
file name is the same as the date of the transaction. These two things make sure that
that document is linked to this specific transaction.</p>
Repeat this process with all the receipts and invoices you have and suddenly you
have a system of organizing (most of) your financial documents next to your financial
data. And it's even version-controlled!</p>
So when your tax advisor asks you for all those documents to file your tax returns,
it'll take you less than 5 minutes to collect everything. Because all you would have to
do is go to the Fava</a> web interface, set the time period to last year, and you'll have
a list of all the relevant transactions with their documents attached. Download
everything and zip it off to your tax advisor. They'll be so proud. 🫶🏻</p>

That's it! That was the entire post, me nerding out about something I find cool.</p>
If you found this post interesting and also like to nerd out about keeping track of
personal finances and about organizing things, I wrote an entire book on this topic</a>
that you might find interesting.</p>
The book assumes that you are comfortable with reading and writing Python code, and
teaches you how you can build your own system to keep track of your personal finances in
a developer-friendly manner using only open-source software. Readers have given it good
feedback so far, so if you're into such nerdy topics, I think you would like it too! 🙃</p>


LangChain applications using Ollama
2023-11-19T00:00:00+00:00
Like almost everyone else in the tech industry today, I've been looking into LLMs
recently. I'm still fairly new to everything but I feel like I'm starting to get a
general sense of how things work. What's inside</em> an LLM is something I'm (at least at
this point) still considering a black box. Regardless, it's fun to play around with
these models and learn more about how they work and how software developers can start
using these in their toolkits.</p>
As part of this, I tried to write a simple LangChain</a> program that lets me
programmatically work with an LLM running on my machine. The code examples shown on the
LangChain documentation default to using OpenAI's ChatGPT, but for whatever reason my
OpenAI API keys keep getting rate limited. I didn't feel like shelling out $20 yet, and
besides, I have</em> an LLM running locally, so why not make use of it.</p>
Fortunately, LangChain can</em> work with Ollama. The following sections describe the steps
I took to get everything working.</p>
1. Install Ollama</h3>
The first thing to do is, of course, have an LLM running locally! We'll use Ollama</a> to
do this. On macOS, the easiest way is to use brew install ollama</code> to install Ollama and
brew services</code> to keep it running.</p>
~/W/l/llms main ❯ brew services start ollama </span></span>
==> Successfully started `ollama` (label: homebrew.mxcl.ollama)</span></span></code></pre>
At this point Ollama should be listening on port 11434 for incoming requests. You can
open up http://localhost:11434/</code> in the browser to double check.</p>
Next, browse through the Ollama library</a> and choose which model you want to run
locally. In this case we want to run llama2</code> so let's ask Ollama to make that happen.
Run ollama pull llama2</code>.</p>
2. Install LangChain</h3>
The next step is to have a Python project with all the necessary dependencies installed.</p>
Initialize a Python project somewhere on your machine, using whatever tools you use. I
personally use poetry</a> to manage Python projects. The following command should take
care of installing all the dependencies.</p>
~</span>/W/l/llms main ❯ poetry add fastapi langchain langserve sse-starlette uvicorn</span></span></code></pre>3. Write the code</h3>
Now that we have all the dependencies in place, let's focus on the code! Here's the code
I used:</p>
from</span> typing</span> import</span> List</span></span>
</span>
from</span> fastapi</span> import</span> FastAPI</span></span>
from</span> langchain.llms</span> import</span> Ollama</span></span>
from</span> langchain.output_parsers</span> import</span> CommaSeparatedListOutputParser</span></span>
from</span> langchain.prompts</span> import</span> PromptTemplate</span></span>
from</span> langserve</span> import</span> add_routes</span></span>
import</span> uvicorn</span></span>
</span>
llama2</span> =</span> Ollama(</span>model</span>=</span>"llama2"</span>)</span></span>
</span>
template</span> =</span> PromptTemplate.from_template(</span>"Tell me a joke about </span>{topic}</span>."</span>)</span></span>
</span>
chain</span> =</span> template</span> |</span> llama2</span> |</span> CommaSeparatedListOutputParser()</span></span>
</span>
app</span> =</span> FastAPI(</span>title</span>=</span>"LangChain"</span>,</span> version</span>=</span>"1.0"</span>,</span> description</span>=</span>"The first server ever!"</span>)</span></span>
</span>
add_routes(app, chain,</span> path</span>=</span>"/chain"</span>)</span></span>
</span>
if</span> __name__</span> ==</span> "__main__"</span>:</span></span>
    uvicorn.run(app,</span> host</span>=</span>"localhost"</span>,</span> port</span>=</span>9001</span>)</span></span></code></pre>
As you can see, there are a few things going on in this code. We first build an Ollama</code>
model object with the model set to llama2. Next we write a simple prompt template with a
parameter called topic</code>. We'll later see how the user can pass a topic to get back a
response from the LLM. Next, we initialize a chain</code> using the LangChain Expression
Language</a>. Finally, we initialize a new FastAPI application and then use
langserve.add_routes</code> to mount the LangServe API routes.</p>
Save this code to a file called main.py</code> and run it using python main.py</code>. This should
start a FastAPI server containing the LangServe endpoint we just defined. On my machine
I see the following message when the app starts:</p>
~/W/l/llms main ❯ python chain.py</span></span>
INFO:     Started server process [25442]</span></span>
INFO:     Waiting for application startup.</span></span>
</span>
 __          ___      .__   __.   _______      _______. _______ .______     ____    ____  _______ </span></span>
|  |        /   \     |  \ |  |  /  _____|    /       ||   ____||   _  \    \   \  /   / |   ____|</span></span>
|  |       /  ^  \    |   \|  | |  |  __     |   (----`|  |__   |  |_)  |    \   \/   /  |  |__   </span></span>
|  |      /  /_\  \   |  . `  | |  | |_ |     \   \    |   __|  |      /      \      /   |   __|  </span></span>
|  `----./  _____  \  |  |\   | |  |__| | .----)   |   |  |____ |  |\  \----.  \    /    |  |____ </span></span>
|_______/__/     \__\ |__| \__|  \______| |_______/    |_______|| _| `._____|   \__/     |_______|</span></span>
</span>
LANGSERVE: Playground for chain "/chain/" is live at:</span></span>
LANGSERVE:  │</span></span>
LANGSERVE:  └──> /chain/playground/</span></span>
LANGSERVE:</span></span>
LANGSERVE: See all available routes at /docs/</span></span>
</span>
LANGSERVE: ⚠️ Using pydantic 2.5.1. OpenAPI docs for invoke, batch, stream, stream_log endpoints will not be generated. API endpoints and playground should work as expected. If you need to see the docs, you can downgrade to pydantic 1. For example, `pip install pydantic==1.10.13`. See https://github.com/tiangolo/fastapi/issues/10360 for details.</span></span>
</span>
INFO:     Application startup complete.</span></span>
INFO:     Uvicorn running on http://localhost:9001 (Press CTRL+C to quit)</span></span></code></pre>
And that should be it! You can now visit http://localhost:9001/chain/playground/</code> to
play around with the LLM interface you just built! Here's a screenshot of the LangServe
Playground I see on my machine:</p>
</p>
It works! And the cool thing is that there are no API keys or anything involved. All the
code necessary for this application is running locally.</p>


Switching between projects in Neovim
2023-10-08T00:00:00+00:00
I've historically used Vimr</a> as my Neovim GUI. It's a pretty nice macOS application
written in Swift that lets me open up multiple projects in separate windows and allows
me to switch between those windows using Cmd-`.</p>
My usual development workflow consists of always having two applications open: Kitty</a>
and Vimr. For each project, I have a terminal session open in a Kitty window and the
source code open in a Vimr window. I constantly switch back and forth between the
two using Cmd-Tab. And every now and then I need to switch between projects which is
where the Cmd-` shortcut was quite handy.</p>
The Problem</h3>
A few months ago, one of the plugins I use (Telescope) started using some functionality
only available in Neovim 0.9+, while Vimr was based on 0.8.2. And it looked like the
Vimr project had gone inactive</a>, which meant that I couldn't update my plugins without
breaking the whole thing. This in turn meant that the ability to switch between projects
using Cmd-` (which is an essential part of how I work) was gone.</p>
So I could either wait for the Vimr maintainers to push an update, or find a different
way to switch between projects. I chose the second option. I replaced Vimr with
Neovide</a> and started looking for plugins that would let me switch between projects
within a single Neovim session easily, essentially replacing Cmd-`.</p>
Workspaces</h3>
One of the plugins I found was workspaces.nvim</a> which lets you add, remove and switch
between workspaces within a single Neovim session. There's also a Telescope integration
so I could set up a keyboard shortcut that would pop up a list of all the available
workspaces and let me switch between them easily.</p>
This worked quite well. The only hiccup was that when I would switch between projects,
all the open buffers from the older project would remain while Neovim's current working
directory would be set to the one from the new project. This meant that every workspace
switch involved manual work of closing the existing buffers and opening up new ones.</p>
Luckily, the author of workspaces.nvim had written another plugin that helped me solve
this problem.</p>
Sessions</h3>
Vim/Neovim has a concept of sessions which represent the current state of your work
(windows, buffers, etc.). The interface isn't the easiest to use as the native commands
feel quite low level. But the author of workspaces.nvim wrote another plugin called
sessions.nvim</a> which provides a nice wrapper around the underlying interface.</p>
After installing sessions.nvim, you can run SessionsSave</code> and SessionsLoad</code> to save
and load session files. So with these two commands in place, what remained was to figure
out how to hook up workspaces.nvim to save a session before a workspace switch and load
a session right after.</p>
Workspaces + Sessions</h3>
Workspaces.nvim allows for "hooks" which are functions or commands you can run when
certain pre-defined events happen. In my case, what I wanted (roughly) was that when I
switch from one project to another, Neovim should:</p>

save the state of the first project to a session</li>
close the existing buffers</li>
switch the current working directory to that of the second project</li>
refresh NvimTree</a> to show the second project's file structure, and finally</li>
load a previously-saved session for the second project into the current window</li>
</ol>
It might sound like a lot of work but actually it was quite easy to represent in code
using a combination of workspaces.nvim and sessions.nvim. The following snippet is what
I ended up using:</p>
require</span>(</span>"workspaces"</span>).</span>setup</span>({</span></span>
    hooks</span> =</span> {</span></span>
        open_pre</span> =</span> {</span></span>
            "SessionsSave"</span>,</span></span>
            "silent %bdelete!"</span>,</span></span>
        },</span></span>
        open</span> =</span> {</span></span>
            "SessionsLoad"</span>,</span></span>
            "NvimTreeOpen"</span>,</span></span>
        },</span></span>
    }</span></span>
})</span></span></code></pre>
This setup gives me exactly what I want. I had to give up Cmd-`, but I can now use
Ctrl-k to bring up a list of available workspaces and switch between them easily.</p>
The open_pre</code> hook is run before</em> a workspace switch, where we save the current state
to a session and close the currently open buffers. And the open</code> hook is run after</em> a
workspace switch, where we first load a previously-saved session and then update the
NvimTree panel. The complete code containing the setup</code> functions of the other plugins
is available in my plugins.lua</a>.</p>

So far I'm quite happy with this solution. I really like that both workspaces.nvim and
sessions.nvim provide a fairly minimal but functional API, particularly how
workspaces.nvim hooks let you bring other plugins into the picture and enable cool
things.</p>
It's quite a pity for Vimr to not be active anymore. It's an excellent application and I
hope the author would be able to spare some time again in the near future to bring it
back. At the moment though, I'm quite happy with the switch to Neovide, and it's a joy
to use the setup described in this post!</p>


Experiences hunting for a software engineering job in 2023
2023-06-21T00:00:00+00:00
At the beginning of May 2023, Shopify decided to lay off</a> 20% of their
workforce. That 20% unfortunately also included me, which meant it was time for
me to look for another job.</p>
So over the past ~6 weeks, I've been interviewing for senior/staff backend
engineering roles almost full time. Here are some raw stats:</p>

In total I interviewed with about 30 companies.</li>
I received a rejection from 13 of them.</li>
About 10 companies I ended up declining myself.</li>
5 of those were "archived". Some of them ended up ghosting me, or they filled
up the position with someone else further along in the hiring process, or
there were internal restructurings in the company and the nature of the
position wasn't clear anymore (which is something I found somewhat
interesting).</li>
Last but not least, I accepted one of those offers.</li>
</ol>
This blog post post is a collection of everything I observed and learned during
this process.</p>
1. There is less hiring overall.</h3>
The first thing that jumped out to me almost right away is that the current job
market (at the time of this writing in June 2023) isn't ideal.</p>
Yes, there are certainly jobs out there. But it feels like there are less jobs
out there than, let's say, 2 years ago. At that time, if you would visit the
careers page of almost any company, it was virtually guaranteed that they were
hiring software developers. At the time of this writing, not so much.</p>
And if you're like me and have specific criteria you're looking to fulfill (eg.
in my case I was looking for fully remote or maximum</em> one day in the office per
week, hopefully having something to do with the Python ecosystem, reasonable
salary, etc.) the list of potential companies reduces even further.</p>
I'm hoping that the situation improves over the next few months. But at least as
of this writing I feel that the market is somewhat reluctant.</p>
2. Startups are more sober when it comes to hiring.</h3>
... and that's a good thing!</p>
When interviewing with startups this time around, it's interesting to hear lines
like "we're not in a rush to hire developers", which is in stark contrast to the
hiring wave 2-3 years ago.</p>
Personally speaking, I'm 100% on board with this way of hiring. Adding
developers to the team just to prove to your investors that you're growing never
felt sustainable in the first place. So now that the economic climate isn't as
good, it seems like early-stage companies are focusing more on basic questions
like "does our economic situation / business model allow for hiring X
developers" or "how many developers do we actually need to build out X in to our
product", which I find rather nice.</p>
As someone who is hunting for a software job, it definitely makes my life
harder. It's still good to know that the overhiring trend is reversing.</p>
3. Unreasonably extensive take-home tests are normal.</h3>
This is something that bothered me the most.</p>
One very common stage in interview processes is a "take-home test" or a "coding
challenge". If you're not familiar, the idea is that the company gives you a
programming problem to work on, a suggested timeframe, and a time limit, and you
have to submit your solution within that time limit. In my experience, these
problems are usually things like "build a JSON API that exposes some business
logic over an endpoint" or "build a command line tool to solve X".</p>
The idea seems appealing, and if done right, it can be a nice way to filter
candidates out. Indeed, some of the companies I interviewed with had nice
take-home tests which left a good impression at the end.</p>
The problem is that a lot of the companies seem to do it awfully wrong.</p>
3.1 The tests take way too long.</h4>
Most of these tests are way</em> too long. And it's not always clear why. There
were times when it felt like the company just wanted me to jump through a 1-2
days long hoop without a specific objective in mind.</p>
There are certainly cases when they're upfront that they expect candidates to
spend 1-2 days preparing the solution. And then there are cases where the
instructions include something like "please don't spend more than a few hours
working on this". In reality though you know</em> that there's no way that that
particular problem can be conceptualized and solved in less than 2 days (if not
more) especially if you want to submit a solution that you personally would be
happy with.</p>
I was interviewing full-time so I was able to spend that kind of time. I
honestly cannot imagine someone with a full-time job (and a life outside of that
job) to allocate that kind of time on such take-home tests.</p>
Hiring managers: if you're reading this, please please please make sure your
coding challenges can be conceptualized and implemented within 2-3 hours. You
want to assess whether the candidate can understand a problem and write good
code, not whether they can clone Facebook in 1 day.</p>
3.2 It's hard to predict how the submission will be evaluated.</h4>
Another thing I noticed is that there's a fair bit of likelihood for the
reviewers and you to not</em> be on the same page regarding how the submission will
be evaluated.</p>
Acceptance criteria is usually specified but either it's vague or the feedback
that you receive (in case of a rejection) may make you feel that the reviewer
was expecting something different, which wasn't clear in the original acceptance
criteria.</p>
As an example, I personally find it very vague when a task description says "the
solution should be production ready". In my mind, this raises so many more
follow-up questions. And even after clarifying those it may not be possible to
implement something to the reviewer's liking. My take-away (after a few
rejections) is that in a lot of cases the interviewers are expecting you to
solve a problem the way they</em> solve problems without explicitly communicating
it to you. How would that work? Your guess is as good as mine.</p>
In another one of the tests I worked on, the task description mentioned "please
don't spend more than a few hours on this task". So I was honest and didn't
spend more than a few hours working on the task. This meant I had to cut
corners. But I was aware of which corners I cut and we talked about those
corners in the follow-up interview.</p>
After the follow-up interview I was given the feedback that the interviewers
felt I didn't spend too much time on the task and hence thought I'm not very
interested in the job.</p>
Wat?</p>
4. LinkedIn is a good way to receive leads.</h3>
One of the first things I did after being laid off was to update my LinkedIn
profile and set the status to "open to work". I chose not to apply the publicly
visible (#opentowork) badge but instead make the status visible to only those in
recruiting positions (dedicated recruitment companies or otherwise).</p>
Just doing that got me quite a few inbound requests that I probably might not
have run into otherwise.</p>
In fact, the company I finally ended up signing with is one that reached out to
me on LinkedIn.</p>
5. There is more competition than normal.</h3>
As a consequence of multiple companies laying developers off one after the
other, there are simply more (very skilled) developers on the market than there
would normally be. This means that companies have too much of a choice. They're
able to reject candidates knowing that there are plenty of other good ones in
the pipeline.</p>
How do I know this?</p>
There were a few companies where my interviews went well</em>. The discussions were
productive. I was able to respond to their questions and tasks in a way that
they were happy with. And I didn't notice any sign that would suggest that my
performance would warrant a rejection.</p>
At the end I still received a rejection with the message that they decided to go
with another candidate who was better than me. 🤷🏻‍♂️</p>

I'm not really sure how to conclude this blog post. Most of my learnings from
interviewing in this climate have not been too positive. So I can only hope that
this blog post didn't come across as too much of a rant. 😅</p>
In any case, if you're a software developer currently in the middle of job
interviews and are frustrated or are close to it, just know that it has much
less to do with your skills as you might think. The job market is just not as
good at the moment and I hope that it improves over the next few months. Also,
acknowledging that job hunting and interviewing (particularly full-time) is
stressful can help.</p>
Good luck!</p>


How to convert multiple images to PNG or JPEG on macOS
2023-01-06T00:00:00+00:00
I recently discovered a neat little feature built into macOS that lets you
convert multiple images from one format to another in one go.</p>
In Finder, select the image(s) you want to convert and right-click (also called
the "two-finger" click) and select "Quick Actions" followed by "Convert Image"
in the submenu that shows up.</p>
</p>
This should open up a new window that looks like this:</p>
</p>
Now, you can select the target format (JPEG, PNG, or HEIF) and the image size
(small, medium, large, or original) and click on the blue "Convert to" button.
And that's it!</p>
In the past, before I switched to using macOS as my daily driver, I always had
to Google for "how to convert image to png in linux" to do this. Some search
results would send me to online conversion websites where I could upload my
images and download the results, which I wasn't interested in for privacy
reasons. The other results would either direct me to webpages that linked to a
list of desktop applications for this purpose, or they would document the
command-line arguments to ImageMagick</a>'s convert</code> command.</p>
Installing desktop applications on Linux wasn't always straight-forward for some
reason. convert</code> would always do the job, but it was a bit of a hassle to
always look up the command line arguments and their positions. Besides, I found
this Google search to be a huge waste of time given that the task isn't that</em>
complicated.</p>
macOS makes it so much simpler.</p>


Automatically balancing Beancount DKB transactions
2022-08-16T00:00:00+00:00
Beancount</a> is one of my favorite packages in the Python ecosystem.</p>
What is Beancount?</h2>
I've written about it in much more detail in other places</a> on the internet, but
if you're new to it, Beancount is a PyPI package that provides a set of tools
for you to keep track of your personal finances using Python.</p>
The entire workflow revolves around a .beancount</code> file on your disk that
contains all</em> your financial transactions (really, "all" as in
"from-the-beginning-of-time all") specified using the double-entry bookkeeping
format. Keeping it up to date involves the following steps:</p>

Download all your financial activity (at regular time intervals) from your
bank's website as PDF/CSV/whatever-your-bank-allows exports.</li>
Convert this data into a format that Beancount understand (this is the part
which may or may not involve custom Python code).</li>
Add/append the output of step #2 to you.beancount</code>.</li>
</ol>
At the end, you.beancount</code> contains all your financial activity where you can
use all the tools that Beancount (and the ecosystem) provides to run all sorts
of analysis on it.</p>
Beancount DKB</h2>
One of my main bank accounts is with DKB</a>. So to get Beancount working with DKB
data exports, I wrote and currently maintain beancount-dkb</a>, which provides a
PyPI package to convert DKB CSV exports to the Beancount format.</p>
So if one of the lines of your DKB CSV exports looks as follows:</p>
"15.08.2022";"15.08.2022";"Lastschrift";"REWE Filialen Voll";"REWE SAGT DANKE.";"DE00000000000000000000";"AAAAAAAA";"-15,37";"000000000000000000";"0000000000000000000000";"";</span></span></code></pre>
... running it through an ECImporter</code> instance makes Beancount output something
like the following:</p>
2022</span>-</span>08</span>-</span>15 *</span> "REWE Filialen Voll" "REWE SAGT DANKE"</span></span>
    Assets</span>:DKB:EC</span>        -</span>15.37</span> EUR</span></span></code></pre>
In terms of double-entry bookkeeping (which is what Beancount relies on), this
is halfway there. We still need to add a second leg to this transaction to
balance it out such that the total sum is zero.</p>
This step is something we usually do manually. But it doesn't always have to be.</p>
Pattern matching against the Payee</h2>
Disclaimer: I stole the following idea from beancount-mercury</a> written by
Michael Lynch</a>.</em></p>
If we look at the original transaction, the payee</a> is always the same. And even
if won't always be the same, it's at least going to contain the same pattern.</p>
We can make use of this fact to tell the DKB importers that if the payee string
contains a specific value, automatically add a second posting to a separate
matching account (which is what we would have done manually anyway). This is
specifically the feature I ended up pushing to PyPI last weekend</a>.</p>
The package distributes two importers: ECImporter</code> (for the German EC accounts)
and CreditImporter</code> (for credit cards).</p>
After the update from last weekend, initializing ECImporter</code> looks as follows:</p>
CONFIG</span> =</span> (</span></span>
    ECImporter(</span></span>
        'DE1234567898765432100'</span>,</span></span>
        'Assets:DKB:EC'</span>,</span></span>
        'EUR'</span>,</span></span>
        file_encoding</span>=</span>'ISO-8859-1'</span>,</span></span>
        # New</span></span>
        payee_patterns</span>=</span>[</span></span>
            (</span>'REWE Filialen Voll'</span>,</span> 'Expenses:Supermarket:REWE'</span>),</span></span>
        ],</span></span>
    ),</span></span>
)</span></span></code></pre>
payee_patterns</code> is a new parameter that should contain a list of (regex, account)</code> tuples as an argument. With this in place, ECImporter</code> compiles the
given regular expressions and uses this list to check if the payee of a given
transaction matches any of the given values. If that is the case, the second
posting (referencing the account</code>) is automatically inserted.</p>
Similarly, the CreditImporter</code> has been adjusted as follows:</p>
CONFIG</span> =</span> (</span></span>
    CreditImporter(</span></span>
        '1234********5678'</span>,</span></span>
        'Assets:DKB:Credit'</span>,</span></span>
        'EUR'</span>,</span></span>
        file_encoding</span>=</span>'ISO-8859-1'</span>,</span></span>
        # New</span></span>
        description_patterns</span>=</span>[</span></span>
            (</span>'REWE SAGT DANKE'</span>,</span> 'Expenses:Supermarket:REWE'</span>),</span></span>
        ],</span></span>
    ),</span></span>
)</span></span></code></pre>
The idea is the same as in ECImporter</code>. Just that instead of payee_patterns</code>,
CreditImporter</code> relies on description_patterns</code> because at the time of this
writing, the CSV exports from DKB credit card accounts do not include a payee
field. So the transaction's description string is all we have to work with.</p>
Result</h2>
With this updated API and the updated importers, beancount-dkb</code> automatically
outputs the following transaction:</p>
2022</span>-</span>08</span>-</span>15 *</span> "REWE Filialen Voll" "REWE SAGT DANKE"</span></span>
    Assets</span>:DKB:EC</span>        -</span>15.37</span> EUR</span></span>
    Expenses</span>:Supermarket:REWE</span></span></code></pre>
The fact that I don't have to add this one line myself reduces my work by about
a half. Thanks Michael</a>, for coming up with the idea!</p>


Replacing GNU Stow with Dotbot
2022-07-24T00:00:00+00:00
One of my guilty pleasures is maintaining my dotfiles</a>. This is one project I
can yak-shave and play with as much as I want without caring about the actual
impact</em> or in general the usefulness of the whole endeavour. So this weekend I
decided to indulge myself.</p>
Part of the process was cleaning up the older dotfiles and updating them to
newer preferences. And the other part (which is the subject of this blog post)
was the installation process, which is basically the process of how these files
get from my dotfiles</code> repository to my home folder.</p>
There are many different ways you can do this. So far, I've been using GNU
Stow</a> for this purpose.</p>
GNU Stow</h2>
Stow is a useful little program to help you manage symbolic links.</p>
Given a source directory and a target directory, it is able to add symbolic
links into the target directory that point back to the source. So if you're
maintaining your dotfiles in a git repository and want to have your home folder
link back to them, Stow is perfect for this.</p>
As an example, assume your dotfiles</code> project contains a nvim</code> folder for your
Neovim configuration files and that it should be linked to ~/.config/nvim</code>,
Stow can do that for you if you run stow nvim --target $HOME/.config</code>.</p>
If you maintain multiple dotfiles, your setup process might end up in a shell
script resembling the following:</p>
mkdir</span> -p</span> ~/.config/kitty</span></span>
stow</span> kitty</span> --target</span> $HOME</span>/.config</span></span>
</span>
mkdir</span> -p</span> ~/.config/nvim</span></span>
stow</span> nvim</span> --target</span> $HOME</span>/.config</span></span>
</span>
stow</span> zsh</span> --target</span> $HOME</span>/</span></span></code></pre>
While this does the job, the one thing that never really sat well with me (no
actual impact</em>, remember?) was the fact that it's a shell script.</p>
Generally speaking, I'm allergic to shell scripts. I don't write them on a
regular basis, so when I do have to work with one, I constantly need to look up
the documentation of the most primitive things (eg. "bash if else syntax") which
is really unproductive. Debugging shell scripts is also not the most fun
activity. So in general, I try to avoid them as long as I can.</p>
Dotbot</h2>
This weekend I was looking for a dotfiles manager (again, no actual impact</em>)
and found dotbot</a>.</p>
Dotbot is a dot</strong>file bo</strong>ot</strong>strapper that takes in a (YAML)
configuration file that describes the symbolic link definitions (and some other
metadata) and updates those definitions on disk. The functionality it offered
looked almost exactly like what I wanted so I decided to give it a try.</p>
The installation feels a bit clunky (the default instructions recommend
installing it as a git submodule) but luckily it's published on PyPI as a
package so we can use pipx</code></a> to install it.</p>
$</span> pipx install dotbot</span></span></code></pre>
The next step was to remove the stow</code> invocations in favor of a dotbot
configuration file. The most recent version of my config file is available
here</a>, but essentially it looks something like this:</p>
-</span> defaults</span>:</span></span>
    link</span>:</span></span>
      relink</span>:</span> true</span></span>
</span>
-</span> link</span>:</span></span>
    ~/.config/kitty/kitty.conf</span>:</span> kitty/kitty.conf</span></span>
    # ...</span></span>
    ~/.zshrc</span>:</span> zsh/zshrc</span></span>
</span>
-</span> create</span>:</span></span>
    -</span> ~/.config/kitty</span></span></code></pre>
The defaults</code> section contains some default configuration options</a> which I'm
not going to write about here as the README does a much better job of explaining
what it is for.</p>
The real work happens inside link</code>, which is basically a key/value map of the
dotfile's destination and the source. This is where you can tell dotbot what
should be linked and where. Lastly, the create</code> section contains a list of
directories that should be created prior to any linking taking place.</p>
With this place, dotbot can be invoked on the command line to set up all the
symbolic links in one go:</p>
$</span> dotbot</span> --config-file</span> install.conf.yaml</span></span>
</span>
Link</span> exists ~/.config/kitty/kitty.conf</span> -</span>></span> /home/siddhant/Work/projects/dotfiles/kitty/kitty.conf</span></span>
...</span></span>
Link</span> exists ~/.zshrc</span> -</span>></span> /home/siddhant/Work/projects/dotfiles/zsh/zshrc</span></span>
All</span> links have been set up</span></span>
Path</span> exists /home/siddhant/.config/kitty</span></span>
...</span></span>
All</span> paths have been set up</span></span>
</span>
==</span>></span> All tasks executed successfully</span></span></code></pre>
This feels much</em> cleaner than the original shell script. Of course, given how
many dotfiles I have (which is not that many), the original script would have
sufficed. But as I mentioned before, none of this is an exercise in usefulness,
so I personally consider this change perfectly acceptable. 🌝</p>
Encrypted files</h2>
The one thing I'm missing from dotbot is support for encrypted files. It would
be nice if you could add encrypted files to the git repository and have dotbot
decrypt them automatically before putting them in place.</p>
Note that there is a shell</code> directive</a> for running shell commands. So in
theory, we could add a shell command for decrypting a sensitive file before the
symbolic links are established. Unfortunately, it seems like these commands are
run after</em> the linking has been done, while we want the decryption to happen
before</em>.</p>
This means that I've been forced to add back a setup.sh</code> file which looks like
this:</p>
# decrypt crestic password</span></span>
age</span> --decrypt --identity</span> ~/.age/keys.txt</span> --output</span> crestic/password crestic/password.enc</span></span>
</span>
# install</span></span>
dotbot</span> --config-file</span> install.conf.yaml</span></span></code></pre>
The first line uses age</code></a> to decrypt a password file and put it in a location
where dotbot can read it and eventually place the link in the right location.
This is less than ideal, but it works.</p>
Conclusion</h2>
Overall I have really come to like dotbot. It does everything I need it to do
and some more.</p>
There is also support for plugins</a> and there are a bunch of nice plugins</a> out
there as well. For instance, you can manage your crontab</code> schedule or specify a
list of apt</code>/snap</code> packages to be installed automatically. While not a strict
necessity, it's nice to know that these things are there.</p>
In any case, I do see myself using dotbot going forward. At least until the next
time I feel the need to yak-shave.</p>


The tools I used to write an ebook
2021-10-04T00:00:00+00:00
After writing and launching a technical ebook, I've had a few people ask me
about the tools I used in the process. Instead of answering individually to
everyone, I thought I would document the entire process here and send a link out
instead.</p>
The Big Picture</h2>
It's easier to talk about the entire process by breaking it down into three
different stages: the input, the pipeline, and the output.</p>
Input is the source code of the book - basically the inner content that the
readers end up reading. The Pipeline is the process I used to generate the final
artifacts ready for distribution. And the Output is how those artifacts end up
in the hands of the readers.</p>
Let's look at them one by one.</p>
The Input</h2>
At its core, the book is a bunch of plain-text Markdown</a> files.</p>
~/W/p/personal-finances-python main ❯ tree book/src/</span></span>
</span>
book/src</span></span>
├── 00-about.md</span></span>
├── 01-introduction.md</span></span>
├── 02-plain-text-accounting.md</span></span>
├── 03-beancount.md</span></span>
├── 04-workflow.md</span></span>
├── 05-testing.md</span></span>
├── 06-reporting.md</span></span>
└── 07-closing-thoughts.md</span></span>
</span>
0 directories, 8 files</span></span></code></pre>
Each chapter is its own Markdown file. There are some code samples and images
that these files reference which are also baked in to the final artifacts. But
when it comes to the "source code" of the ebook, that's about it. Nothing
special there.</p>
I really like Markdown because (for me) it strikes exactly the right balance
between size and expressiveness. The language is simple and small enough that
you don't have to make a lot of effort keeping the syntax in your head. This is
in stark contrast to something like RST</a> where it's really hard to remind
yourself what primitives produce what output. At the same time it's powerful
enough that you don't feel like you can't express something - at least most of
the time.</p>
Because of this reason, most of my writing for the past few years has ended up
being in Markdown. So that's what I ended up using for the ebook too.</p>
The Pipeline</h2>
This is the slightly more interesting part where I have both good and not so
good things to say.</p>
I used a program called pandoc</a> to convert the Markdown files into PDF and ePub
files. Pandoc is a swiss-army knife of document conversion. You can take a
document in one format and have pandoc convert it to a different (supported)
format by passing it through pandoc.</p>
In my case, the input format was Markdown and the final output was PDF and ePub.
While the ePub conversion was a breeze, the PDF formatting took quite some time
to get right (more on that later).</p>
The good</h3>
Pandoc is extremely powerful. The process of converting documents from one
format to another using just one command sometimes felt like cheating.</p>
The manual</a> is also very useful and comprehensive. In my experience, if Pandoc
is capable of feature X, it's extremely likely that you'll find it in the
manual.</p>
The package is relatively straight-forward to install on most UNIX systems. For
PDF generation, you'll most likely have to install a TeX distribution in
addition. If installing TeX using a package manager is not really your taste,
there are also official docker images that you can use.</p>
Pandoc also supports filters</a> which lets you do useful things with the output.
For instance, I used the pandoc-fignos</a> filter to add numbers to figures and
reference them in the document. This was really useful to have text in the PDF
link to the image.</p>
Lastly, pandoc has been around for quite some time and has seen quite a bit of
usage. This means that if you're running into an issue, it's very likely that
someone else already ran into it so there might be a relevant Github thread or a
StackOverflow question, maybe also with a solution. This makes it easier to get
a headstart in case you're stuck on something.</p>
The not so good</h3>
The not so good was mainly getting the PDF formatting right. Like I mentioned
before, the ePub generation was a breeze. But getting a presentable PDF was a
massive pain.</p>
At its core, Pandoc is a document converter. It's not an ebook formatting tool.
This means that while technically you can</em> generate ebook PDFs, Pandoc doesn't
know that it's working with an ebook (unless of course you're generating an
ePub file).</p>
This means that the default PDF output is really plain / bland. If you want the
formatting to look a certain way, expect to spend some time</a> and effort</a> to
get things running. In my case, I ended up spending days</em> looking through the
pandoc manual and searching for the web to fix specific formatting issues I was
running into.</p>
How do you properly insert a cover image at the front? How do you format
blockquotes so they look a specific way? How do you specify the header and
footer texts? What all listings</a> settings do you need to adjust so that code
blocks look good</em> and consistent with inline code?</p>
These are all questions I wish I wouldn't have had to spend time figuring out
answers to. But unfortunately I did. If you're proficient with LaTeX, you might
have a different experience because pandoc uses LaTeX to generate PDFs from
Markdown. However, the last time I touched LaTeX was about 7-8 years ago so as
of this writing, I wouldn't really call myself proficient there.</p>
So almost every single issue I ran into was a bit of a pain. I would do an
online search, find a solution, which was often in the form of a LaTeX snippet
which I was supposed to put in the original source's preamble. Over time, all
those LaTeX snippets added up and it became really tricky to figure out if a new
issue was because of some weird combination of the existing snippets or some
other reason entirely.</p>
At some point I gave up trying to fine-tune these settings by hand and started
looking for a boilerplate instead. In hindsight, this should have been step 1.
😁</p>
I found an excellent Pandoc / LaTeX template</a> which I ended up using for my
ebook. This almost instantly solved all the formatting issues. If you're writing
an ebook and are planning to generate a PDF using Pandoc, I highly recommend
this template. Most of the formatting issues you could care about can be
controlled using variables in this template. And the final output looks very
nice.</p>
The Output</h2>
There's honestly not much to write in this section.</p>
I chose Gumroad</a> to distribute the final ebook artifacts and I really like
them. 😀</p>
Gumroad is an excellent service which lets you sell digital products. It's
mostly intended at online creators. Whatever product you want to sell, you can
upload it and set up a product page for it, and can start selling immediately.
You can get paid using either a PayPal account or direct bank transfer,
depending on the country you're living in. And they even take care of handling
taxes for you.</p>
If you're considering selling a digital product, make sure to give them a try.</p>
Conclusion</h2>
That's about it! I think those are pretty much all the technologies I used to
build and sell my ebook.</p>
If there's anything you'd like to know more about, feel free to @/DM me on
Twitter!</p>


How you can track your personal finances using Python 🐍
2021-08-29T00:00:00+00:00
In this post, I'd like to describe how you can track your personal finances
using a workflow that is highly focused on data privacy, is 100% self-hosted,
and uses only the Python ecosystem.</p>
I'm also secretly hoping that some of you find this interesting enough to
explore it in more detail</a>. 🙂</p>

We all know that money is important. Having money buys us freedom and lack of
money is stressful. So how do we know whether we're doing good on that front?</p>
The answer is simple: we track our money.</p>
We keep an eye on how much money is coming in to our accounts, how much is going
out, and when and how often those events are happening. We keep an eye out on
trends, what are the recurring expenses, how many of those are necessary, so on
and so forth.</p>
There's a good chance that you're doing this already. There are plenty of
off-the-shelf solutions one can pick from for this purpose. There are mobile
and SaaS apps that can connect to all your bank accounts, import all your
financial transactions, and show you consolidated data.</p>
Not that there's anything particularly wrong with such apps. I'm instead of the
opinion that my financial data (from across all my bank accounts) is something
only I</strong> should have consolidated access to. Financial data is one of the most
privata data I own. So limiting the possible attack vectors sounds to me like
an obvious choice.</p>
If you're out searching for such software and limit your search to open-source
only solutions, you're most likely going to come across Plain Text Accounting</a>,
which is something I'll describe in this post.</p>
Big (and Quick) picture</h2>
TL:DR</strong>: Maintaining double-entry accounting</a> based records of your financial
transactions in plain-text files is the way to go.</p>
Double-entry accounting is a great way to track your finances. In this system,
the flow of money between accounts is represented using transactions. You can
think of a transaction as an "entry" of sorts that talks about a specific
instance of money flowing between accounts. In most cases, transactions are
composed of two "legs", where one leg is the credit</strong> side and the other one
is the debit</strong> side.</p>
One of the most important rules of double-entry accounting is that the sum of
the amounts of the individual legs in a transaction must be zero. A transaction
is said to be "balanced" if this rule is satisfied.</p>
Here's a sample transaction to help you visualize this:</p>
2021</span>-</span>01</span>-</span>01 *</span> "AMAZON.DE"</span></span>
    Assets</span>:MyBank</span>    -</span>42.00</span> EUR</span></span>
    Expenses</span>:Amazon</span>   42.00</span> EUR</span></span></code></pre>
This transaction represents an Amazon purchase I made where the money was
deducted from one of my accounts (called Assets:MyBank</code>) and added to an Amazon
expense account.</p>
A collection of such transactions is what makes up your financial ledger.</p>
Double-entry Accounting ♥️ Python = Beancount</h2>
The Python ecosystem contains a really neat package called Beancount</a>.</p>
Beancount is a command-line implementation of the double-entry accounting
system that works on top of plain-text files. It mainly provides the following
three things:</p>

basic primitives for working with money (eg. data structures like Accounts,
Transactions, Postings, etc.)</li>
a language specification for defining financial transactions in a plain-text
format (the snippet I showed earlier is valid Beancount code)</li>
command-line scripts that tie everything together</li>
</ol>
As with most Python packages, getting started is as easy as creating a virtual
environment and running pip install beancount</code>.</p>
What does the ledger look like?</h2>
I wrote earlier that one of the main things that Beancount provides is a
language specification for defining financial transactions in a plain-text
format.</p>
What does this format look like? Here's a quick example:</p>
option "title"</span> "Alice"</span></span>
option "operating_currency"</span> "EUR"</span></span>
</span>
; Accounts</span></span>
2021</span>-</span>01</span>-</span>01 open Assets</span>:MyBank:Checking</span></span>
2021</span>-</span>01</span>-</span>01 open Expenses</span>:Rent</span></span>
</span>
2021</span>-</span>01</span>-</span>01 *</span> "Landlord" "Thanks for the rent"</span></span>
    Assets</span>:MyBank:Checking</span>     -</span>1000.00</span> EUR</span></span>
    Expenses</span>:Rent</span>               1000.00</span> EUR</span></span></code></pre>
What you see here is a very basic financial ledger of an imaginary person.</p>
In the first few lines we specify some metadata. Then we open (initialize) two
Accounts. And the last three lines define a transaction where rent was deducted
from one of those accounts and debited to the other one.</p>
Note that the account names don't always have to correspond one-to-one to
real-world accounts. You can define as many accounts as you like, each for a
specific purpose. For instance, you can have one account to track your
supermarket expenses, one for rent, one for Netflix, so on and so forth.</p>
As I said, this is a very basic example. Real-world ledger files tend to be much
longer.</p>
For instance, at the time of this writing my personal .beancount</code> file contains
close to 21,000 lines.</p>
~</span>/Work/finances main ❯ wc -l goel.beancount</span></span>
20996</span> goel.beancount</span></span></code></pre>How does the data flow?</h2>
I hope that at this point you have a basic understanding of Beancount and
double-entry accounting. In this section, I'll quickly run you through how data
flows in such a system.</p>
The core idea with Beancount is that the user is responsible for storing all
their financial transactions inside their .beancount</code> file. This file acts as
the single source of truth for all your financial data from all your banks.</p>
So how do your transactions from your bank(s) end up in this file? In a series
of three simple steps:</p>
1. Download transactions from your bank</h3>
Almost every bank allows you to export your data in some form. You should be
able to log in to your bank's website, select a timeframe, and download all the
transactions in a given file format. This usually tends to be CSV, but you can
also download a PDF or sometimes an OFX</a> file.</p>
This is the first step in a Beancount-driven workflow: download all your
original data to your machine.</p>
Note that ideally this should be something that Python can parse without much
ceremony. For instance, if I would have the choice between CSV and PDF, I would
most likely pick a CSV download. Not because Python cannot parse PDF data
(because it can</a>) but because CSV is a much simpler data format than PDF.</p>
2. Convert transactions to the Beancount format</h3>
The next step is to take these CSV files and convert this data into a format
that Beancount understands.</p>
Beancount provides an importer framework</a> to help with this process. Importers
are Python programs which take a source file (eg. the downloaded CSV), parse it,
and convert the data into data structures that Beancount provides. A text
representation of these data structures is what makes up your personal ledger.</p>
Now, obviously Beancount does not know about all the banks on the planet and
what their CSV exports look like and how they should be parsed. This is where
the Python developer inside you can shine.</p>
The Importer</code> classes shipped with Beancount are more like protocols. The base
class is literally called ImporterProtocol</code>. They define method signatures and
leave the implementation up to you. This way, you can inherit the base
ImporterProtocol</code> class for your own importer, override the relevant methods,
and let Beancount know of the existence of these importer classes using a
configuration file.</p>
from</span> beancount.ingest.importer</span> import</span> ImporterProtocol</span></span>
</span>
def</span> convert_to_beancount_data_structs</span>(line):</span></span>
    """</span></span>
    Implement me!</span></span>
    """</span></span>
</span>
class</span> MyBankImporter</span>(</span>ImporterProtocol</span>):</span></span>
    def</span> extract</span>(self, file):</span></span>
        """</span></span>
        Read and parse the file and convert the original data into Beancount</span></span>
        data structures</span></span>
        """</span></span>
        entries</span> =</span> []</span></span>
</span>
        for</span> line</span> in</span> file</span>:</span></span>
            entries.append(convert_to_beancount_data_structs(line))</span></span>
            </span></span>
        return</span> entries</span></span></code></pre>
Depending on what banks you have accounts with, you can define multiple such
importer classes and let Beancount know of their existence in a configuration
file. Everything else will be handled for you automatically using the command
line scripts included in Beancount.</p>
3. Balance the transactions</h3>
The next step is also the most fun part (at least I find it fun).</p>
We take the output of the previous step, pipe everything over to our
.beancount</code> file, and "balance" transactions.</p>
Recall that the flow of money in double-entry accounting is represented
using transactions involving at least two accounts. When you download CSVs from
your bank, each line in that CSV represents money that's either incoming or
outgoing. That's only one leg of a transaction (credit or debit). It's up to us
to provide the other leg.</p>
This act is called "balancing".</p>
For instance, assume that the output of the previous step was the following
transaction:</p>
2021</span>-</span>01</span>-</span>01 *</span> "Landlord" "Thanks for the rent"</span></span>
    Assets</span>:MyBank:Checking</span>    -</span>1000.00</span> EUR</span></span></code></pre>
The total amount of the legs in this transaction does not yet sum to zero. It's
up to us to balance it.</p>
Balancing involves figuring out what the transaction is about and assigning it
an equal and opposite leg. In this case we can see that this transaction has to
do with us paying rent. So the second leg should clearly contain the
Expenses:Rent</code> account.</p>
2021</span>-</span>01</span>-</span>01 *</span> "Landlord" "Thanks for the rent"</span></span>
    Assets</span>:MyBank:Checking</span>     -</span>1000.00</span> EUR</span></span>
    Expenses</span>:Rent</span>               1000.00</span> EUR</span></span></code></pre>
This process of balancing is something we do for all the transactions, so that
the entire ledger can be marked as balanced.</p>
4. That's it!</h3>
That's basically it. Those three steps, performed at regular intervals of time,
should ensure that your .beancount</code> data contains all your financial
transactions and is in good shape.</p>
I prefer doing this every month. So on the first Sunday of every month, I
prepare a fresh cup of coffee, download all the CSV files, run them through my
importers, and balance all the unbalanced transactions.</p>
You might think that this is too much work. I have to admit that I had a similar
thought when I started out.</p>
In my experience so far, it has been the exact opposite. The entire process has
never taken me more than 45 minutes to finish. Considering that I do this once
a month, the time investment seems more than fair. And the added benefit is that
by balancing these transactions by hand, I get a fairly good idea of what was
happening in my accounts in the previous month.</p>
Visualization</h2>
So far we talked about entering data into the system.</p>
The real power of personal accounting software is in showing you insights from
that data. In this section I'll quickly walk you through the two tools I end up
using the most: bean-query</code> and Fava.</p>
1. bean-query</code></h3>
bean-query</code> is a command line utility shipped with Beancount that lets you run
SQL-ish queries on your financial data. The query language is specific to
Beancount. But if you're familiar with SQL, you'll feel right at home with
bean-query</code>.</p>
Here's an example run from my own data where I want to know how much I spent on
public transport in the last three years, grouped by the year:</p>
beancount> SELECT \</span></span>
        year, SUM(number) AS total \</span></span>
        WHERE account ~ 'Expenses:PublicTransport' AND year >= 2019 \</span></span>
        GROUP BY year</span>;</span></span>
year total </span></span>
---- ------</span></span>
2019 672.60</span></span>
2020 328.02</span></span>
2021  30.50</span></span></code></pre>
The first column in the output contains the year and the second one contains the
total amount of money that I spent buying public transport tickets.</p>
Side note: I've been working remote since 2016, but it's pretty clear from those
numbers when the COVID-lockdown hit my town the most.</p>
2. Fava</h3>
Fava is a community-maintained web interface for Beancount. At its core it's a
Flask application which reads your .beancount</code> file and provides all sorts of
visualizations you'll find useful.</p>
Here's a screenshot of one of the reports that Fava can generate for you:</p>
</a></p>
There are multiple reports built-in, for instance balance sheet, income
statement, document browser (another neat feature of Beancount where you can
attach documents (eg. invoices) to transactions), and more. And</em> you can
filter all the data using timeframes.</p>
There is an online demo at this URL in case you'd like to get a feel for it:
https://fava.pythonanywhere.com/</a>.</p>
Between Fava and bean-query</code>, I use Fava the most. I feel that the
visualizations provided by Fava give me a really good sense of almost all the
financial insights I'm looking for.</p>
Conclusion</h2>
That was it! Congratulations for making it this far! I hope this article was
able to interest you in this privacy-focused way of tracking personal finances.</p>
If you're interested in learning more, you should consider buying an ebook I
wrote on this topic titled Tracking Personal Finances using Python</a>.  In this
ebook, I explain the concepts mentioned above in detail to help you build your
own personalized multi-banking application using Python.</p>
Beancount and Double Entry Accounting can be slightly confusing to folks who are
just starting out. So this ebook is my attempt at lowering the barrier to entry.</p>
And if you have questions about any of this, feel free to reach out to me on
Twitter</a>!</p>


How to debug better
2021-07-22T00:00:00+00:00
The very act of writing software implies that we (unintentionally) introduce
bugs. It's possible to reduce the number of bugs you introduce. But it's
definitely not possible to bring it down to zero. This means that most software
we work with build or use will fail, at times in interesting ways. And then you
the software developer would have to go in and figure out where things are going
wrong.</p>
After spending a few years building, breaking, and then fixing software, I've
found a couple of things to keep in mind that help me when I'm trying to debug
something. They work on all kinds of bugs - from broken libraries where there's
a difference between documented and actual behavior, to distributed systems
where touching one component affects a different one in ways that one would
normally not expect.</p>
In this article, I'd like to describe some of these approaches that I've found
useful in discovering the root causes of issues in production and how to resolve
them.</p>
1. Read and understand what's happening</h2>
The first approach is also the most obvious one.</p>
We need to read, and more importantly, understand any and all the information
available to us. And when I use the word "understand", I mean really</em> give it a
long and hard look and try to get to the bottom of it.</p>
When you're trying to fix something that's broken, it's critical to build a
mental model of what the expected behavior is and what is happening instead.
Once you have that in place, reading and understanding the information available
to you (in the form of an error message or a log entry, or whatever you may have
at hand) goes a long way in figuring out what exactly is going wrong.</p>
Often, I talk to engineers who are stuck with some exception in their code,
unable to make progress. Almost in every single case I've noticed that they tend
to ignore reading the stack trace that they are looking at. The general practice
is to put the entire exception inside the Google search box, with the hope of
finding a StackOverflow response with the exact same exception. At times this
works. When it doesn't though, it's pretty much a dead-end strategy of
debugging.</p>
2. Think using first principles</h2>
Software is generally</em> pretty deterministic (note the emphasis on the word
generally</em>).</p>
Things tend to have a well-defined flow. There are obviously huge exceptions to
this rule. But if your work involves developing user-facing products on the web
(like mine), you're most likely working at an abstraction level where the flow
tends to be pretty well-defined.</p>
As an example, a SaaS product generally has a frontend component, a backend
component and then the infrastructure that the whole thing is running on. There
are other things as well. For instance you likely have a load balancer layer
that all user-traffic goes through before hitting the frontend.  Or you might
have some queues which the backend is using to perform some background
processing.</p>
Each of those components has a specific purpose in this chain. If, for instance
you're facing a bug that spans across multiple layers, it's generally helpful to
think from first principles as to what each component is supposed to do and
what it might be doing instead.</p>
For instance, if you notice that the data the backend is receiving is not
something that it's supposed to, it'll help to build a mental model of the
complete user request - what the frontend sends to the backend, if the load
balancer can modify the request headers in some way, stuff like that.
Visualizing the stack like this goes a long way in being able to pinpoint the
problematic layer(s).</p>
3. Avoid trying out random things</h2>
One of the rather interesting things I've seen software developers (including
me) do when trying to fix a problem is trying out random</em> things. I will try
to illustrate this with a very basic example. Consider the following function:</p>
def</span> add</span>(a, b):</span></span>
    return</span> a</span> -</span> b</span></span></code></pre>
We have a function called add</code> which is supposed to add both its arguments
and return the result. Instead, the function returns the subtraction as a
result. What's I've seen folks often do is to try out random approaches. One
example would be changing the definition to something like:</p>
def</span> add</span>(a, b):</span></span>
    return</span> a</span> *</span> b</span></span></code></pre>
... in the hopes that the final result will be what we expect.</p>
This urge is natural. And at times this may certainly work. But it's a bit like
throwing spaghetti at the wall. It's not particularly efficient. It's much
better to spend a quiet moment with the code at hand and try to get to the
bottom of it. The success rate will be much higher that way.</p>
Again, I've been guilty of this myself as well. There are times when in a
desperate attempt to fix things, I try out solutions that are more or less
random relative to the problem at hand. These are definitely times when the "hey
I wonder what happens if I do this</em>" thought process kicks the rational part of
the brain out. Over time though, I've noticed that this is not particularly
effective. So now whenever I feel this urge coming up, I try to keep it in check
and start by really trying to understand what's wrong.</p>
4. Ask someone else</h2>
One of the best things to do is to ask someone! This could be a coworker, or
someone on the Internet in some online community you're a part of.</p>
Asking a coworker is definitely the more helpful option though, because
coworkers are likely to have more context on what you're working on. Ideally
this is someone with more experience than you. Senior Engineers have ran into
enough problems over their careers. And over time they develop an ability to
pattern-match problems. So they might look at your bug which might remind them
of something different they were working on a few years ago (if not the same).
Having someone give you this perspective is invaluable and can go a long way in
solving the problem at hand.</p>
Of course, this is not to say that only senior folks can help. Good engineers,
regardless of whether they're junior or senior, tend to ask very interesting
questions. They might help you see the problem from a different angle, or follow
a line of questioning that you weren't even considering earlier. Such
conversations are super helpful.</p>
And if you don't have any of those available, you can always try talking to a
rubber duck</a>. The first time I tried it out, I was extremely surprised at how
effective it was.</p>
5. Step out and take a walk</h2>
If you've done all of the above and still don't know how to proceed, the best
way forward might just be to take a break. Sometimes our brains have had enough.
When you reach that point, there's really no way that you can consume and/or
process more information.</p>
In such situations, just stop working. Get away from your desk, ideally go out
and get some fresh air. Go for a run. Jump in the shower. Anything that keeps
you away from the problem at hand so that your body gets the break it needs.</p>
You may feel guilty about not "working" when there are problems to solve. But
what's really happening is that you're letting your brain work in the
background. So technically speaking, you're still making progress. And sooner or
later, the answer will hit you out of nowhere.</p>


Uploading binary wheels to PyPI from Github Actions
2020-11-03T00:00:00+00:00
If you write software using Python and your code has third-party dependencies,
you've probably heard of "wheels" before.</p>
Wheels are the current standard of distributing Python packages and are intended
to replace eggs. Wheels are part of what makes the magic happen when you pip install</code> something.</p>
Background</h2>
Let's say you want to install Flask to build a web application. One of the steps
fairly early on in your development cycle is going to be pip install Flask</code>.
When you run pip install Flask</code>, pip</code> queries the Python Packages Index (more
commonly known as PyPI</a>) and fetches the Flask package to install it for you
locally.</p>
The file format that pip</code> downloads is what we're going to talk about in this
post. Historically, this has been eggs (because you know, snakes lay eggs).
These days though, this format tends to be either a source distribution (which
is basically the raw source code of the package) or a "wheel".</p>
If you'd like to get a deeper understanding of what wheels actually look like at
a file-level, I would recommend reading PEP 427</a>. In this post though, we'll
focus on how</strong> to build wheels. Specifically, how to build binary wheels on
Linux, macOS, and Windows for Python packages that contain C extensions.</p>
The Problem</h2>
I maintain a package called streaming-form-data</a> which allows you to parse
multipart/form-data</code> encoded byte chunks. Due to the nature of this package,
the performance-critical parts were written using Cython. So whenever I wanted
to create a new release on PyPI, I would compile the Cython code to a C
extension and include that in the final distribution.</p>
Long story short, this package contains a C extension.</p>
So far, I had just been uploading source distributions to PyPI. This meant that
the end-user was responsible for ensuring that they have a working C compiler
(something like GCC) on their system. It mostly worked, until recently when
someone reported that they found it a hassle to install it on Windows because
the compiler they were using had some issues with the generated C code (or the
other way around, one could argue).</p>
At that point I thought about looking into generating platform-specific binary
wheels for this package and uploading those to PyPI alongside the source
distribution. This meant generating wheels for Linux, macOS, and Windows which
users could install directly on their machines without having to compile
anything. The advantage for users was huge, in that they didn't have to worry
about system-level dependencies at all.</p>
Building platform wheels</h2>
This is where things got slightly tricky. To generate a wheel for a given
platform, you need to have access to that platform. I run Linux</a> on my private
laptop, which meant I could only generate binary wheels which would work on
Linux boxes but not on macOS or Windows. In theory, I guess there might</strong>
have been a possibility of generating these wheels for other platforms using
virtual machines. But that was a huge</strong> "might".</p>
Luckily, I discovered that Github Actions</a> actually supports macOS and
Windows</a> runners, which made the whole thing so much more convenient.</p>
I was already using Github Actions for continuous testing. So building platform
wheels was a matter of adding another workflow to the CI setup.</p>
Roadblock #2</h2>
My first attempt was to build the platform wheels by using the wheel</a> package.
If you have wheel</code> installed, you'll have a bdist_wheel</code> command available
under python setup.py</code>. Going that route, the Actions workflow definition
looked something like the following:</p>
build_wheels</span>:</span></span>
  runs-on</span>:</span> ${{ matrix.os }}</span></span>
  strategy</span>:</span></span>
    matrix</span>:</span></span>
      os</span>: [</span>ubuntu-latest</span>,</span> macos-latest</span>,</span> windows-latest</span>]</span></span>
  steps</span>:</span></span>
    -</span> uses</span>:</span> actions/checkout@v2</span></span>
</span>
    -</span> uses</span>:</span> actions/setup-python@v2</span></span>
      with</span>:</span></span>
        python-version</span>:</span> 3.8</span></span>
</span>
    -</span> name</span>:</span> Setup pip</span></span>
      run</span>:</span> |</span></span>
         python -m pip install --upgrade pip</span></span>
         python -m pip install wheel</span></span>
</span>
    -</span> name</span>:</span> Build wheel</span></span>
      run</span>:</span> python setup.py bdist_wheel</span></span>
</span>
    -</span> uses</span>:</span> actions/upload-artifact@v2</span></span>
      with</span>:</span></span>
        path</span>:</span> ./dist/*.whl</span></span></code></pre>
This would've been it, except that it only worked for building wheels on macOS
and Windows, but failed on Linux. The reason was that building binary wheels on
Linux is further complicated by the fact that there are hundreds (if not
thousands) of Linux distributions in existence, each treating system packages
differently.</p>
There's the manylinux</a> project from the PyPA that aims to make this easier. But
running it directly still feels (to me) a bit of a hassle.</p>
Add to this the fact that building binary wheels would have to take into account
multiple CPU architectures, building packages for different Python versions
across multiple operating systems. The solution described above would've
required much more work to be "production ready".</p>
The Final Solution</h2>
Luckily, I came across cibuildwheel</a>.</p>
cibuildwheel</code> is an amazing project that helps with exactly this problem. It's
a Python package that's meant to run on your CI server (Github Actions in this
case) and abstracts away all the details of building binary wheels for multiple
platforms for different CPU architectures.</p>
So instead of relying on python setup.py bdist_wheel</code>, you use cibuildwheel</code>
instead to build the wheels. Since this looked really promising, I modified the
"Build wheel" section in the previous snippet to look like the following:</p>
build_wheels</span>:</span></span>
  runs-on</span>:</span> ${{ matrix.os }}</span></span>
  strategy</span>:</span></span>
    matrix</span>:</span></span>
      os</span>: [</span>ubuntu-latest</span>,</span> macos-latest</span>,</span> windows-latest</span>]</span></span>
  steps</span>:</span></span>
    -</span> uses</span>:</span> actions/checkout@v2</span></span>
</span>
    -</span> uses</span>:</span> actions/setup-python@v2</span></span>
      with</span>:</span></span>
        python-version</span>:</span> 3.8</span></span>
</span>
    -</span> name</span>:</span> Setup pip</span></span>
      run</span>:</span> |</span></span>
         python -m pip install --upgrade pip</span></span>
         python -m pip install cibuildwheel==1.6.4</span></span>
</span>
    -</span> name</span>:</span> Build wheel</span></span>
      run</span>:</span> python -m cibuildwheel --output-dir dist/</span></span>
      env</span>:</span></span>
        CIBW_BUILD</span>:</span> cp36-* cp37-* cp38-*</span></span>
</span>
    -</span> uses</span>:</span> actions/upload-artifact@v2</span></span>
      with</span>:</span></span>
        path</span>:</span> ./dist/*.whl</span></span></code></pre>
So basically just replacing the python setup.py bdist_wheel</code> step with
cibuildwheel</code>, which made the workflow go through without a hitch. And at the
end of the pipeline, there were 12 binary wheels available for me to upload to
PyPI.</p>
Always love it when software "just works". 😅</p>
The only modification I had to make was specifying a value for the CIBW_BUILD</code>
environment variable to restrict the Python versions. For this specific case, I
limited the Python versions to 3.6, 3.7, and 3.8. Other than that though, I left
everything in cibuildwheel</code> to their default values.</p>
Conclusion</h2>
Python being an interpreted language, if any part of your source code needs
compilation, things get slightly tricky if you want others to be able to install
your packages easily.</p>
Binary wheels are a huge step in that direction, but the underlying complexity
of the problem they're trying to solve means that the user experience of
building wheels for different platforms is often not 100% smooth.</p>
IMHO though, cibuildwheel</code> eliminates much of that complexity for you, the
packager. The solution I described earlier works pretty well for me, and I would
highly recommend using it in your own projects if there's the need.</p>


The Almanack of Naval Ravikant
2020-10-24T00:00:00+00:00
If you're even remotely into reading books on self-improvement or "modern"
philosophy, The Almanack of Naval Ravikant</a> is the one book you absolutely must
consider reading.</p>
It's available online for free. Physical copies cost a little bit of money
(obviously). But given the value that I feel is inside the book, the price is
extremely</strong> reasonable.</p>
I first learned about Naval from his interview on the Tim Ferris podcast</a>. It's
always a treat to listen to someone who looks at things from a first-principles
point of view. Plus he has this rare ability to express a ton of insight using
an easy to understand language. In the sense that when you've listened to what
he's saying or have read one of his blog posts or tweets, sometimes you go "that
makes too much</em> sense".</p>
So when I came to know that this book is available on Amazon, I ordered it right
away. And already a few pages into the book, I had the feeling that this is
going to be one of my all-time favorite books. Mostly centered as a guide to
wealth and happiness, the contents of this book cover a lot of ground. At no
point though does that impact the core ideas that this book is trying to convey.
There really are a ton of things to learn from in this book.</p>
Content-wise, the book comprises of some of Naval's best tweets (and the
"tweetstorms"), essays and interviews. The flow from one topic to the next is
very fluid and you should easily be able to finish it in a day or two. But even
though it's an easy read, you'll most certainly find that this book requires
multiple readings to be able to fully grasp and internalise the ideas expressed.
Like how it is with most classics.</p>
TL;DR</strong>: read it!</p>


Continuous Deployment with Netlify and Zola
2020-09-13T00:00:00+00:00
I spent some time last week enabling continous deployment for this site. As a
quick bit of background, I use the Zola</a> static site generator to generate the
site and Netlify</a> to host it.</p>
I can highly recommend both the tools, as Zola is a very simple and fast
generator and Netlify is a pretty feature-complete static hosting platform with
an extremely</strong> generous free tier. And as we'll later see, the two work quite
well together.</p>

The basic idea is that Netlify allows you to connect your sites to Git
repositories. At the time of this writing, this includes Github, Gitlab, and
Bitbucket. Once a repository has been connected to a site, every time you push
to the "main" development branch, Netlify will trigger a deployment pipeline.</p>
This workflow can either be configured from within their web interface or using
a netlify.toml</code> config file in the root folder of your repository. For static
sites using Zola, this file looks like the following.</p>
[</span>build</span>]</span></span>
publish =</span> "public"</span></span>
command =</span> "zola build"</span></span>
</span>
[</span>build</span>.</span>environment</span>]</span></span>
ZOLA_VERSION =</span> "0.12.0"</span></span></code></pre>
The most important line of code in this config is setting ZOLA_VERSION</code>. This
tells Netlify that we're working with a Zola site so it should please ensure
that the zola</code> binary exists, corresponds to version 0.12.0</code> and is available
on $PATH</code>. The other two lines of config tell Netlify that on every push to
master</code> (or whatever your main development branch is called), the command zola build</code> should be run and the contents of the public</code> folder (which zola build</code>
populates) should be published to the static site.</p>
Once this file is in place and your Git repository is connected to your Netlify
site, the process basically just works. That's basically the only switch you
would have to flip.</p>
Overall this process was quite straight forward and I'm really happy with the
result. The nicest thing is that the one manual step involved with this site (of
uploading to Netlify) is not there anymore. Prior to having this pipeline in
place, I was using netlify-cli</code> every time I made a change to deploy the site.
Additionally there was also the dance of local git branches to be able to
differentiate between what's live (the stable</code> branch) and what's not (the
master</code> branch).</p>
Now, I can just make some edits and push to Github and the end result is
automatically published to https://sgoel.dev</a> within a few seconds. Netlify
even sends me a nice little email everytime the site contents change. I don't
have to perform any extra steps locally, and there's only one git branch to
maintain. And all the stuff that's pushed to Github is "live", and everything
else is a draft.</p>
💯/💯 would recommend.</p>


Multiple Mixins (and naming conflicts) in Python
2020-06-28T00:00:00+00:00
Mixins are quite useful. In case you're not familiar with the idea, it's a step
in the direction of "composition over inheritance".</p>
In Object Oriented Programming, consider that you're implementing a class and
you want a bunch of functions to be available inside of that class. The idea
behind mixins is to implement these functions in other independent classes and
inherit your actual class from these utility classes. Such utility classes are
called "mixins".</p>
In Python, for instance, this is possible due to multiple inheritance</a>.</p>
As an example, assume that we want to implement an Animal</code> class, the objects
of which should have the capability to both "bark" and "meow". Putting
biological concerns aside, this can be implemented by defining the "barking"
functionality inside a Barker</code> class and the "meowing" functionality inside a
Meower</code> class. You can then inherit the Animal</code> class from both Barker</code> and
Meower</code>.</p>
class</span> Barker</span>:</span></span>
    def</span> bark</span>(self):</span></span>
        print</span>(</span>'Woof!'</span>)</span></span>
</span>
class</span> Meower</span>:</span></span>
    def</span> meow</span>(self):</span></span>
        print</span>(</span>'Meow!'</span>)</span></span>
</span>
class</span> Animal</span>(</span>Barker</span>,</span> Meower</span>):</span></span>
    def</span> greet</span>(self):</span></span>
        self</span>.bark()</span></span>
        self</span>.meow()</span></span></code></pre>
This has the advantage that if later on we want to add another Animal</code>-ish
class which wants the ability to "bark", we simply have to choose Barker</code> as
one of the base classes to have the bark()</code> function available in the new
class.</p>
While mixins come in super handy when solving real-world problems, they often
come with their own set of issues. And some of these issues come from how the
programming language you're using implements multiple inheritance.</p>
In the Python world, one such problem is — what happens if two different mixins
define the same function?</p>
Let's take a real-world-ish problem to illustrate the issue.</p>
Consider that we're writing a web application serving HTTP requests. At the end
of every web request we would like to perform some cleanup related to the
database and the cache. Let's also assume that our web framework of choice
provides an on_finish</code> hook on individual requests to place such cleanup code.</p>
When implementing something like that using mixins, this is how it could roughly
look like.</p>
class</span> DatabaseCleaner</span>:</span></span>
    def</span> on_finish</span>(self):</span></span>
        print</span>(</span>'Cleaning the database'</span>)</span></span>
</span>
class</span> CacheCleaner</span>:</span></span>
    def</span> on_finish</span>(self):</span></span>
        print</span>(</span>'Cleaning the cache'</span>)</span></span>
</span>
class</span> Request</span>(</span>DatabaseCleaner</span>,</span> CacheCleaner</span>):</span></span>
    pass</span></span></code></pre>
For this code snippet, what do you think the output would be?</p>
Because of the way Python implements multiple inheritance, it ends up calling
the function defined in the first</strong> base class. In our example, this happens
to be DatabaseCleaner.on_finish</code>, which is what ends up being called. This has
the unfortunate side-effect that CacheCleaner.on_finish</code> never gets called.</p>
This is obviously not what we want. We want to print both "Cleaning the
database" as well as "Cleaning the cache".</p>
One solution for this problem is to name those on_finish</code> functions
differently. So that DatabaseCleaner</code> implements on_finish_database</code> and
CacheCleaner</code> implements on_finish_cache</code>.</p>
While that could work (with a little bit of extra effort), it sounds a little
odd.  What if those two classes are written by two completely different authors
who don't know about the existence of each other? Not to mention that this now
puts the responsibility on the calling code to remember to call those
on_finish_*</code> functions, defeating the mixin-magic in the first place.</p>
Luckily, Python provides a way to call the "next in line" function, which refers
to the function with the same name in the "next" base class. Let's modify the
clean()</code> functions in both the mixins to something like the following.</p>
class</span> DatabaseCleaner</span>:</span></span>
    def</span> clean</span>(self):</span></span>
        print</span>(</span>'Cleaning the database'</span>)</span></span>
</span>
        try</span>:</span></span>
            next_clean</span> =</span> super</span>().clean()</span></span>
        except</span> AttributeError</span>:</span></span>
            pass</span></span>
        else</span>:</span></span>
            next_clean()</span></span></code></pre>
Here, inside the try</code>/except</code> clause, we first try to find the
function that's "next in line" by using super()</code>. In case one was found, we
simply call it. In case no such function was found (which most likely means that
this is the last base class), then super().clean()</code> would raise an
AttributeError</code>, in which case we can choose to do nothing and do a clean exit.</p>
While this does add 6 extra lines to the mixin code, I feel it's a small price
to pay. As library authors, it's nice to act as good citizens and keep in mind
that there are other libraries out there as well which our users might use which
may result in name clashes. I feel it's a good idea to make our code resistant
to such things.</p>
I came across this problem when writing tornado-sqlalchemy</a>, which is a Python
package that provides SQLAlchemy integration for Tornado projects. This
integration is provided using a DatabaseMixin</code> which makes database Session</code>
objects available in the request classes.</p>
When using this library in a work project which also had Sentry</a> integration
enabled through sentry-python</a>, I noticed that both tornado-sqlalchemy</code> and
sentry-python</code> provided (at least at that time) mixins to perform cleanup at the
end of the request. tornado-sqlalchemy</code> did the database cleanup while
sentry-python</code> did some cleanup related to Sentry.</p>
But since Tornado provides only a single cleanup function called on_finish</code>,
the same function was being overridden by both the libraries. This resulted in
the fact that if a user was using both the mixins in their application code,
depending on which class order they used when defining the (multiple)
inheritance, one cleanup function would be skipped completely.</p>
I've since patched the DatabaseMixin</code> provided by tornado-sqlalchemy</code> so that
this problem doesn't exist anymore. But it was still interesting to run into
this issue and find the solution.</p>


Book Review: The Phoenix Project
2020-04-15T00:00:00+00:00
Every now and then I like to look for book recommendations on places like Hacker
News and Lobsters where the audience is primarily technical. The one book that
comes up consistently in these recommendations is The Phoenix Project</a>.</p>
So last week I decided to order it and check for myself what the fuss was all
about. Luckily it arrived at my doorstep right before the Easter weekend, so I
had the entire weekend to read it.</p>
I managed to finish reading over the weekend, and I have to say that this is
easily one of the best books I've ever read</strong>.</p>

“If you don’t like to read, you haven’t found the right book.” – J. K. Rowling</a></p>
</blockquote>
Background</h2>
The book is set inside a fictitious company called Parts Unlimited. It's
narrated from the point of view of Bill, an IT manager. The company is losing
customers and money, the board is really unhappy with the performance, and the
internal business processes are dysfunctional at best.</p>
In this setting, Bill is suddenly promoted to being the VP of a
less-than-optimal IT department and asked to deliver on the so-called Phoenix
Project within 90 days, failing which the entire department shall be outsourced.</p>
The story of how Bill tries to understand his new job description, identifying
and fixing the existing (broken) processes, the people he now has to work and
deal with, navigating the corporate hierarchy so he can get things done, and
even figuring out what needs to be done in the first place, is what the book is
all about.</p>
While reading, I tended to focus on the gaps that development and IT departments
were facing. But as you read more, you find out that the business problems that
this company is facing are much deeper. One prime example is that the upper
management doesn't consider software development / IT as an integral part of the
business. This often leads to unreasonable expectations from other departments
that IT then has to deliver on, clashes between people and departments, lots of
blame games, so on and so forth.</p>
Style</h2>
I wasn't exactly sure what to expect from a "fiction novel about DevOps". DevOps
is probably one of the last subjects that comes to my mind when I think about
fiction. This book proved me wrong in so many ways.</p>
The Phoenix Project is a very well-written book. The storyline is gripping. At
least for me it was difficult to put the book down. And if you've spent a lot of
time working in or with software / IT departments, you'll find that the
characters and situations in this book are quite familiar (if not relatable).
Finally, the message that this book is trying to highlight is delivered to the
reader slowly but consistently.</p>
Should you read it?</h2>
It depends on what your taste in books is like. I do believe that if you work
directly in software development / IT or you interact with other folks who do,
this book is an absolute must-read.</p>
More generally speaking, my recommendation depends on the kind of software
company you work at. In terms of the development / IT split, I feel that there
can be two kinds here.</p>

companies where there is a clear distinction between development and IT</li>
companies where DevOps is core to the software development culture</li>
</ol>
If you belong to the first group, this book is highly recommended. Speaking from
personal experience, I've had a chance in the past to work at a place where the
deployment culture was very much still developers "throwing code over to the
wall" to the operations people. If I had a chance to go back in time and make
everyone there read this book, I absolutely would. This "development /
deployment" distinction affected my personal productivity quite a bit, and I
think having everyone read this book would've at least pushed things in the
right direction.</p>
On the other hand, if you belong to the second group, you may not need</em> to read
this book but it still could be a nice read. If you've worked at companies which
embrace DevOps as a core cultural value, you probably can't imagine having it
any other way. So along those lines, this book can be a humorous tale about what
can happen when things aren't structured that way.</p>

All in all, a wonderful read. Highly recommended!</p>


Following news 24x7 considered harmful
2020-04-13T00:00:00+00:00
One of the most impactful changes I've made to my lifestyle is not following
news 24x7, either from TV channels or aggregator sites. Instead, I read one or
two credible and objective news outlets on the Internet once (or maximum twice)
a day to keep myself updated. But that's about it.</p>
This one practice has had a significantly positive impact on my mental
well-being, and I recommend everyone to try it out, unless your job depends
directly on consuming / generating news.</p>
Quest for Content</h2>
One of my biggest issues with "always on" news is the quest for content. If
you're a content publisher on the Internet pushing content out on a regular to
semi-regular basis, you probably know what I'm talking about. You have to
generate new content all the time</em>.</p>
Unless content generation comes naturally to you, there's a significant effort
that goes in towards generating high-quality content. Not putting in that effort
leads to quality gradually declining. I've seen plenty of websites that started
out extremely focused, publishing very high-quality stuff. Eventually though,
they started running out of stories and started putting out irrelevant and
low-quality content.</p>
I feel that some version of that applies to a couple of news media outlets as
well.</p>
For instance, there are plenty of active news channels in India that started out
on a good note back in the late 90s / early 2000s. But now in 2020 when I look
at the kind of stuff that they're reporting, it's beyond disappointing. Not only
are the stories irrelevant and sensationalist, most of them are extremely low
quality. Check the #WhyIsIndianMediaStupid</a> hashtag on Twitter for examples.</p>
Apart from low quality content, the bigger problem I see is opinionated
reporting. News, in the true sense of the word, is supposed to mean objective
reporting of facts. Ideally it's supposed to be black and white. This does not
seem to be the case anymore.</p>
Now it's completely normal for anchors to deliver news along with their own
opinions, that may potentially promote partisanship. Plus there are enough
outlets reporting non-factual stories ("alternative facts", if you will). Not
only do I feel that this "news -> opinion" transition is a recipe for disaster,
it could also be threatening the very concept of democracy as we know it.</p>
Breaking News</h2>
Most news outlets these days report stories as soon as they come in. A deeper
analysis of why this has become such a huge trend is beyond my understanding. I
do believe that there's some amount of FOMO</a> (Fear Of Missing Out)
capitalization in there.</p>
In any case, what I've noticed is that for someone like me, whose job does not
directly depend on news, the majority of those stories can wait. For me, it
doesn't make a huge difference if I consume those stories right away, or
postpone reading them to the next day or day after.</p>
Attaching the word "breaking" to a given piece of news creates a sense of
urgency in my mind. Whether the urgency is positive or negative depends on the
story. But as of April 2020 it's difficult to find positive news stories, so
most of the times the urgency tends to be negative. This activates all the
anxiety muscles in my body (and I'm pretty sure they're a thing), making me feel
restless, anxious, and completely distracted from what I was doing earlier.</p>
On the other hand, if I don't consume those stories immediately and schedule
reading them to a fixed part of the day instead, this helps at least with the
distraction. I'm probably still going to feel restless and anxious if the story
is negative. But if I restrict myself to reading the news in a given time slot
during the day, at least there wouldn't be any productivity impact during the
rest of the day.</p>
Anxiety</h2>
Like I mentioned earlier, unfortunately most of the news as of April 2020 is
negative. That's just how the world seems to be running its course right now.</p>
And as tiny cogs in this machine, there's not much we can do in the short
term</em> to turn the tide. There are definitely small steps that individuals can
take right now for change to become visible a few years down the line. But right
now, for example, apart from washing my hands and keeping distance from people I
don't know, there's not much else in my capacity to be able to change the
COVID-19 situation across the planet.</p>
So it's more or less a given that when you choose to follow negative news 24x7,
you're essentially choosing to expose your mind to negative news constantly, all
day long. No wonder it's going to push the anxiety levels high.</p>
A core principle of Stoicism</a> is to not worry about what you can't change. This
is much easier said than done, but it's definitely not impossible. But while
you can attempt to train your mind along this line, why not instead solve the
problem at its core and reduce your constant exposure to 24x7 news altogether?</p>

I guess one of the things it boils down to is consuming content on your own
schedule rather than on someone else's. The news outlets you're following,
excepting a few, are most likely driven by an advertising based business model,
which means they want to have as much of your attention as possible. While that
makes them money, it impacts your health.</p>
Besides, extremely important news stories (one example would be official
statements from the local / national governments) have a way of somehow reaching
you wherever you are. So reducing your news consumption frequency is most likely
not going to affect your daily life too much.</p>
I would highly recommend trying this out. So far I've noticed only positive
effects, without impacting my general awareness of what's going on around me.</p>


How to enable target="_blank" links in Zola
2020-04-07T00:00:00+00:00
Zola is a nice static site generator. In fact, so nice that I moved this site
over from Pelican a few weeks back</a>.</p>
The one thing I was missing though was the ability to set target="_blank"</code> on
links within the content. There is a relevant Github issue</a> from last year
which proposes this feature for Zola. But the general consensus from that thread
is that this functionality should ideally be supported by pulldown-cmark</a>,
which is the Markdown rendering library that Zola uses.</p>
While I do agree with that decision, the relevant issue</a> from the
pulldown-cmark</code> issue tracker looks a little discouraging. It is referenced by
a different issue titled Extensible rendering prototype</a> which is currently
marked as closed with a comment</a> from the maintainer saying that it's unlikely
for there to be much progress on this in the foreseeable future.</p>
I wasn't sure what to make of the whole thing, but I do understand how hard it
can be to prioritize open-source projects in between daily work and personal
life. And since my Rust skills at the moment are beginner-level (at best), I
tried to work my way around it using JavaScript instead.</p>
The first step was to define a configuration option called enable_target_blank</code>
to control this setting. Zola allows arbitrary user-defined configuration
options, so this was rather straight forward.</p>
[extra]</span></span>
enable_target_blank</span> = true</span></span></code></pre>
I then added the following JavaScript snippet in the base HTML template.</p>
var</span> enableTargetBlank</span> = function</span>() {</span></span>
    var</span> parents</span> =</span> document.</span>getElementsByClassName</span>(</span>'with-target-blank'</span>);</span></span>
</span>
    for</span> (</span>let</span> parent</span> of</span> parents) {</span></span>
        var</span> links</span> =</span> parent.</span>getElementsByTagName</span>(</span>'a'</span>);</span></span>
</span>
        for</span> (</span>let</span> link</span> of</span> links) {</span></span>
            if</span> (link.hostname</span> !==</span> window.location.hostname</span> &&</span> link.target</span> ===</span> ""</span>) {</span></span>
                link.target</span> =</span> "_blank"</span>;</span></span>
            }</span></span>
        }</span></span>
    }</span></span>
};</span></span>
</span>
{</span>% if</span> config.extra.enable_target_blank</span> %</span>}</span></span>
enableTargetBlank</span>();</span></span>
{</span>%</span> endif</span> %</span>}</span></span></code></pre>
This code snippet picks all the <a></code> elements on the page which are direct /
indirect descendants of any elements containing the with-target-blank</code> CSS
class. For all such links, we set the target</code> attribute to _blank</code>, in case
the links are external and if the target</code> isn't set to something already. And
all this code is defined in the base template so that it's also available in all
the other templates (due to template inheritance).</p>
As a result, you can now add the with-target-blank</code> class on container elements
in which you want all the links to open in a new tab. On this site, for
instance, I do this only on elements containing Markdown rendered from Zola.
Here's the relevant portion from the page.html</code> template that I use.</p>
<</span>div</span> class</span>=</span>"row"</span>></span></span>
    <</span>div</span> class</span>=</span>"col with-target-blank"</span>></span></span>
        {{ page.content | safe }}</span></span>
    </</span>div</span>></span></span>
</</span>div</span>></span></span></code></pre>
Note that for this to work, there are two pre-requisites.</p>

JavaScript needs to be enabled.</li>
You should be able to modify the template code. I'm not sure if you can put
custom code in off-the-shelf themes that Zola provides. But since I
use a custom theme for this site, I didn't have to look too much into this.</li>
</ol>
Other than that though, this approach is quite straight-forward and works
exactly like one would expect.</p>


Building products for yourself
2020-04-05T00:00:00+00:00
I'm a reasonable human being. I say that with a lot of confidence because I
don't think I've ever wanted to build an Uber for squirrels (and I hope that in
the future, I may never want to).</p>
I say this in the context of "building things". You're a bright software
developer. One day you notice that you have a personal itch that you want to
scratch. And then you realize that scratching the said itch could actually help
a lot of other people. So you decide to spend your evenings and weekends working
hard to build something.</p>
We all know that feeling of pure excitement. You spend hours looking for a
project name (and ensure that the .com domain is available). You mkdir -p</code> a
new directory, run git init</code>, and take a screenshot of the root commit (just in
case your product ends up making you a millionaire). There's nothing like it.</p>
Validating ideas</h2>
While all that is exciting, the only problem is that you don't know if someone
else really</strong> would find your product useful. At this point, you only have an
idea, which by itself isn't worth much unless you find someone else who would
like to use it and potentially even pay for it.</p>
This is why the indie software</a> circles stress on "idea validation" so much.</p>
Before you commit to spending the next 6 months working on something, it makes
sense to check if there's even a market. Besides, it's possible to validate
ideas quite cheaply. You can search online for similar products or post on
online forums asking your potential customers if they would be interested in
your product. You can put up landing pages and collect emails. Some people go
even one step further and ask for pre-payments to make absolutely sure that this
is something for which people would be willing to pay.</p>
While validating ideas is definitely good advice, I feel that too much of idea
validation can also sometimes get in the way.</p>
Solving your own problems</h2>
A different approach is to focus on solving your own problems.</p>
Going back to the fact that I'm a reasonable human being, it's very likely that
the problems I face are also legitimate. This in turn means that people who are
similar to me might be facing a similar problem, and hence might benefit from
the solution I build.</p>
This approach makes things slightly easier. Instead of going through a formal
idea validation process, you can just bounce it off of a few friends of yours to
do some basic sanity checks. And if that goes well, you can proceed directly
towards the "building" stage.</p>
What's also nice about this approach is that since you're building for yourself,
you're essentially going to be a customer. This means that you'll have access to
constant customer feedback to help you shape the final product.</p>
Note that we didn't completely get rid of idea validation. We're still
validating ideas, but there's no formal process. We're not spending time setting
up landing pages or collecting emails (and if you're in the EU, worrying about
the privacy aspects of collecting emails) or other such activities. We're
essentially trusting our gut feeling to be right.</p>

This post is not an attempt to make blanket statements in favor of or against
idea validation. Most of the times you should definitely validate your ideas.
You don't want to spend 6 months of your time and all your savings, working on a
product that has no market.</p>
Instead, the message I want to convey is that setting up landing pages is
sometimes advertised as a rite of passage of sorts. Sure, you do want to make
sure that your product would have customers willing to pay for it. But at times,
it's also OK to just trust your gut feeling, do a basic sanity-check on your
idea, treat yourself as the customer, and then immeditately get to work.</p>


Moving from Pelican to Zola
2020-02-11T00:00:00+00:00
Last weekend I moved this site over from Pelican</a> to Zola</a>. The motivations
were based partly in merit, but partly also just for fun.</p>
The migration itself was mostly painless. The part that took the most amount of
time was just converting the posts from Pelican format to Zola format. Even
though both expect Markdown, I found that there were some minor differences here
and there (the front-matter syntax, for example). But nothing that a quick
Python script could not automate.</p>
Pelican / Zola</h2>
I've been using Pelican for this site for about 4 years now. When I started out,
I chose Pelican because it was (and probably still is) the most popular static
site generator written in Python.</p>
And I still think that Pelican is an excellent way of building static sites. You
get a lot of features out of the box. The documentation is excellent. And if you
need a particular feature that hasn't been implemented yet, it's easy to write a
plugin.</p>
I'm not sure if there's any specific disadvantage I can point to. Overall,
Pelican has been super nice to work with. I do know that people often complain
about themes, but I never had to muck around with installing third-party themes
so I can't confirm how valid/invalid those complaints are.</p>
Looking back, I think I switched mostly because I've been trying to pick up
Rust</a> in my free time. And I have this weird theory that the more I surround
myself with tools written in the language I'm trying to learn, the faster I'll
be able to pick the said language up.</p>
There may or may not be some truth to that statement. At least there's a direct
analogy to learning languages that humans use to communicate. In the case of
machine languages though, the argumentation in my head is kind of arbitrary and
the effectiveness remains to be seen. Besides, this site is not some
"real-world" project generating revenue, so I don't necessarily have to weigh
the trade-offs between development time and product impact. "Playing around"
is perfectly fine for a site like this.</p>
So last Friday I went on to StaticGen</a>, and picked up literally the first SSG
from the list that was written in Rust. This happened to be Zola.</p>
Installing Zola</h2>
The installation "process" was basically just downloading the latest release
from Github, and moving the included binary file over to ~/.local/bin</code>. Having
spent slightly more than a decade in the Python world where you have to do the
virtualenv</code> dance every now and then, this was a nice surprise. To be clear, I
don't think I've had huge problems with virtualenv</code>. Combined with Poetry</a>, I
still feel it's a reasonable way to manage Python applications. But it's still
hard to beat static binaries.</p>
I think the installation would've been even quicker if my distro would have had
Zola in the package repositories, but that would be a bit too optimistic of
Debian stable. 🤷‍♂️</p>
Speed</h2>
One of the next things I was pleasantly surprised about was speed</strong>. The fact
that Zola is written in Rust means that even without any particular
optimizations, it's fast. Like, stupidly</strong> fast.</p>
Incremental builds are almost always under 5ms, and building the entire site
from scratch takes ~100ms. That's a huge difference from Pelican where even
incremental builds can easily take 2-3 seconds.</p>
The Migration</h2>
Overall, the transition was fairly painless. The Zola documentation is pretty
clear on how to get started and organize things. So the majority of my time was
spent on converting the blog posts from Pelican format to Zola format (or
rather, writing a script that does that conversion).</p>
While they both use Markdown, the front-matter syntax is different. Plus, tags
in Zola are defined as explicit taxonomies using a TOML-like syntax within the
front-matter, while in Pelican you can define tags directly like how you define
other fields.</p>
I ended up writing a quick Python script to take the content/</code> directory from a
Pelican site and convert all your posts to a format that would be compatible
with Zola. This saved me a lot of time.</p>

Siddhant Goel

Self-hosting static sites on a Caddy-powered VPS

What's next for txtcv

PostgreSQL on Kamal: Deployment, Configuration, and Backups

10 years of personal finances in plain text files

HTMX and Django make status polling trivial

Building a CV builder

Notes from my first marathon

Good Lazy, Bad Lazy: AI usage reflections

Camp 2: Good Lazy</h3> The second (more interesting) camp is when I'm being plain lazy, but of the good kind!</p>

Starting fatherhood

Trying out Zed after more than a decade of Vim/Neovim

1. Vim mode</h3> Yep, that's literally the first reason. After using Vim for 15 years, my fingers have built up enough muscle memory that not using Vim keybindings when writing code is downright impossible.</p>

6. Fast!</h3> Last but definitely not the least is speed! Zed is very</em> fast. Everything feels very snappy, and it's clear that the team has put a lot of effort into making everything fast.</p>

Building Cython (or C) extensions using uv

Moving from Beancount 2.x to 3.x

Handing over the reins of GeniePy

Experiences using Tailwind CSS

One of the most useful directories on my laptop

LangChain applications using Ollama

3. Write the code</h3> Now that we have all the dependencies in place, let's focus on the code! Here's the code I used:</p>

Switching between projects in Neovim

Experiences hunting for a software engineering job in 2023

How to convert multiple images to PNG or JPEG on macOS

Automatically balancing Beancount DKB transactions

Replacing GNU Stow with Dotbot

The tools I used to write an ebook

How you can track your personal finances using Python 🐍

2. Fava</h3> Fava is a community-maintained web interface for Beancount. At its core it's a Flask application which reads your .beancount</code> file and provides all sorts of visualizations you'll find useful.</p> Here's a screenshot of one of the reports that Fava can generate for you:</p>

How to debug better

Uploading binary wheels to PyPI from Github Actions

The Almanack of Naval Ravikant

Continuous Deployment with Netlify and Zola

Multiple Mixins (and naming conflicts) in Python

Book Review: The Phoenix Project

6. Fast!</h3>
Last but definitely not the least is speed! Zed is very</em> fast. Everything feels very snappy, and it's clear that the team has put a lot of effort into making everything fast.</p>