I woke up early morning today and took some minutes to read this blog post about engineering delivery metrics while grabbing my early morning coffee :coffee:.

It provides good insights about how to measure the efficiency of your process from a Pull Request being opened until the code changes are shipped to the production environment.

Without going into low-level details, these are the main things that should happen during that process:

Delivery pipeline

The concepts described in that blog post are something that has been running as a background thread in my mind over the last few months, but I am always struggling to invest some time in it.

Until today! I paired with ChatGPT and I hit two targets with one shot.

I was curious about the first part of the diagram above: how long does it take for a Pull Request to get merged in my current project?

I asked:

First question

And GPT replied:

First answer

This was a good way to start our relationship for this matter :blush: It helped me to avoid:

  • checking which python library should I use.
  • check the github / python library API to understand how to obtain each pull request and its status.

I realized my question was misleading, as it interpreted that I was interested in a specific Pull Request:

Second question

And GPT replied:

Second answer Second answer

I was so impressed that I didn’t even consider testing the script, I wanted more!

Third question

For the sake of readiness, I’ve put its answer in an external gist.

And its explanation:

Third answer

It hadn’t in mind that I was curious about the last six months of activity, and it decided to query Pull Requests from a specific date (yesterday).

So I asked again:

Fourth question

And its answer in an external gist.

It described again the solution, but to be honest I didn’t read it.

At this point, I’m not sure if there’s been a big gain in productivity. Perhaps the major advantage is that I didn’t need to invest some time reviewing a couple of third-party dependencies, but at the same time, I feel I’m trusting blindly what I’m being told.

Unfortunately, things became a bit tedious:

Fifth question

It apologized for the confusion (so cute :cat:) and pointed to the mistake very fast (incredibly fast!)

Fifth answer

This got me very confused and intrigued. I could understand that ChatGPT knew about since and until attributes in the get_pulls() method if they had existed in previous versions of the PyGithub library and got deprecated for whatever reason in the last version of the library. But that was not the case, they have never existed. So I have no idea why ChatGPT decided to use since. My two theories are that it exists in other libraries and learned from that, or it extrapolated other arguments of the REST API and adapted its knowledge about the python wrapper. Quite fascinating IMO even though it was wrong this time.

This is the script associated with the response above. Here GPT made a rookie mistake. We are interested in the last 6 months of data, so as soon as one Pull Request is older than that, we can stop iterating over previous data (pull requests are order by time desc).

I got a couple of additional errors:

Sixth question

This script is interesting because it forgot to add the last line of the script:


This reminded me to don’t trust blindly the machine for now. So I read the script for the first time :laughing: and realized we were retrieving data from the master branch. I can understand that because it used to be the default branch name some years ago, but that’s not the case anymore :punch:.

I suggested using main as the default branch instead:

Seventh question

It was very polite, I was liking it a lot:

Seventh answer

Going back to my previous problem, it was still happening!

Eighth question

And its answer and the updated script, this time with the last line included!

Eighth answer

Here we got an issue that should be easy to spot with the usual linter in your IDE:

Ninth question

And it acknowledged the mistake and provided an updated script.

I was getting a very ugly chart with no data, so I thought maybe we should plot “hours to merge” instead of “days to merge”:

Tenth question

It explained briefly the required changes:

Tenth answer

This is the generated script

At this point, I think both of us needed a break :laughing:.

The chart was still looking pretty bad, so I did what I usually (try to) do in my day-to-day work: read the code, understand what it’s doing, add some helpful debug logs, and find the root cause of what’s going on.

Eleventh question

This was still very interesting but was not increasing my productivity anymore. This is its answer:

Eleventh answer

I was shocked :laughing: It indeed understood the issue (we were using datetime instead of date to create the buckets) but suddenly it introduced numpy (np.) in our script.

Twelfth question

This is its answer and the script using numpy.

Twelfth question

I tested it and it worked :tada:. In less than one hour we had a script to grab the Pull Requests from a specific repo and plot a histogram with the number of hours it took to merge each Pull Request.

I think the job is still not ended, but it was a good start and I needed a break, so I asked it something different to have some social time together:

Last question

I left home at 11:00 am and the cyclist arrived at 12:30 pm. I was lucky I didn’t trust it blindly.


« Home