Test & Code

Brian:
Adam Johnson, welcome back to Test and Code.

Adam:
Thank you for having me.

Brian:
It's like we just talked or something.

Adam:
Feels like it's been minutes, yeah.

Brian:
Yeah, so this episode is going to come out a week later, but it's recorded at the same time. But we're going to talk about a different topic. Last time we talked about pytest-django, but today we're going to do a little Git, right?

Adam:
A little bit of Git that can help you test your code, yes.

Brian:
Yeah, so a part of this is in celebration of a book that you've got called Boost Your Git DX. Now, is this a new book?

Adam:
This is a newly updated book. It's my second update in January. I released it in 2023 and just keeping track with versions of Git, there's about six feature versions of Git a year. Most of it's very low-level details that you'd only care about if you are running GitHub or GitLab or similar hosts. But there are some nice quality of life improvements that come in. And so the latest version of the book, which is all about quality of life improvements with Git, now covers the new stuff.

Brian:
Okay. Quality of life stuff is important. I'm using a lot more Git now lately. And I guess I've been using Git for years, but I've been using more different projects that use different styles. So it sometimes takes a while to get used to the different style of a new team. But anyway.

Brian:
Well, what should we talk about with this Git stuff? Any cool tricks for people?

Adam:
I have cherry-picked, one could say, three different techniques that you can use with writing tests and running tests to integrate Git a little bit better there.

Brian:
Perfect. Some sneak peeks. And these are things that are covered in your book also?

Adam:
Yes.

Brian:
Okay. Okay. Well, before we get going, you brought up cherry-picking, and that's something I've never understood. So maybe, what does that mean?

Adam:
Cherry picking is like taking a single commit, like picking it out of one branch and putting it on another. So a classic example would be like you're running away on some feature branch, you've had a few different commits, but one of them is really just an unrelated bugs fix or documentation fix. So you want to put that into a separate feature branch, separate pull requests up on GitHub. So you go back on your main branch, you start a new feature branch, and then you run get cherry pick. And that commit hash, the commit hash, of the one that you wrote over on the first branch, and then that will pull just that commit over into a new branch.

Brian:
And when you say just that commit, and I don't know the internals will get that much, I just use it. Does it just take that change? So it doesn't just copy all the files that you touched, right? It just takes the change of whatever you fixed.

Adam:
Maybe? Exactly. It just takes the change from that commit. Okay.

Brian:
That's pretty cool. Because there's oftentimes where we've got a branch that we're hardening, like getting ready for a release and you harden a branch. And then there'll be critical bug fixes on there that we want to do both there and in the main branch. so cherry pick might help us with that okay okay absolutely sorry to like completely divert that so let's let's talk about the first tip that you got okay.

Adam:
So git stash have you ever used this brian.

Brian:
Yeah i love well i kind of it's sort of an interesting name because i use it as get throw away um so right.

Adam:
You can use it for like i'll stash it away and maybe i'll use it later.

Brian:
Like Who knows?

Adam:
Like the cable drawer, like you've got 50 commits in your stash. Like maybe it'll be useful one day. But yeah, the idea of stash is to take changes that you have right now and store them away in what's basically a secret commit in their backgrounding git. So they're gone from your working copy. And then you can bring them back by like popping them from the stash. It actually acts like a stack. So every time you run stash on its own, you're pushing onto the stack. and then later you can run pop and undo one of those secret commits the last one that you did.

Brian:
Okay, but they get names or you can name them so you can enlist them. So if you want one out of the middle, you can grab that.

Adam:
Exactly, yeah. They're numbered, yeah.

Brian:
Okay.

Adam:
So the little tip here for using it with tests is that you can create a stash out of just some changes. And the way I would use this is I'm writing some bug fix. I find the code that needs changing. I change that code to work. and now that I've understood the problem, I go and write a test but I want to check that the test would fail before the bug fix and only pass afterwards, so you could go back to the source file and undo the bug fix, whatever it was but if it's a bit more involved, that could be quite annoying so you can run git stash and then just the path to that file like example slash something.py and then just the code change will be gone the test will still be in place You can run the test, verify that it fails, then stash pop, and your code fix comes back, run the test again,

Adam:
it passes. Great. This is a good test.

Brian:
That is a brilliant use of git stash. I love it. You can pretend that you're doing test-driven development even when you've tested after.

Adam:
Exactly. TDD with time travel, right?

Brian:
Yeah. It's what? TDD WTT. I love it. Okay. So that's pretty cool. Okay. That was our first tip, even though we kind of snuck in the whole lesson on what? I don't even remember what we talked about. The other lesson.

Adam:
Yeah, that was tip zero.

Brian:
Zero is cherry picking. Okay. Tip one, git stash. What do we got next?

Adam:
Okay. So git diff. You've probably run to see what has changed. It has a little secret dash dash name dash only flag, name only. It just lists the name of the files that have changed.

Brian:
Okay, so if I run, I've used it with arguments. So if I just run git diff by itself, what is it doing? Is it comparing my changes to the last commit or the head? Or what is it doing?

Adam:
Last commit, exactly, also known as head, yes. So it just shows you what you have changed that you haven't staged with git add yet.

Brian:
Okay, and then name only will just tell me which files I've modified.

Adam:
Exactly. So you can throw on the end of that a path spec, which is Git's name for Globs. And you can filter that list of names. And so the one I would use for this tip is star test star. So show me all the files that have changed that have the word test somewhere in their path. And then you can run pytest with the results from that. You can wrap it in, you know, in ZSHs, a dollar brackets. So run pytest on all the change test files.

Brian:
Just run the change tests. I'm going to be using this every day now. That's awesome.

Adam:
Yeah, definitely, like, cut down your test runtime when you've done, like, refactoring on, like, five or ten files. You want to run those. You can't be bothered to, like, list them out individually. This is the tip for you.

Brian:
Yeah so what is my current process my current process is using like VS Code or something and selecting the test files if I can remember which ones, which ones I've modified or I think it highlights them or something I can do that but this is cleaner, I like it I'm going to put this in a little macro in my bash RC file or zshell RC.

Adam:
Cool idea You can also combine it with some of the other options from GitDiff. So you could pass it a commit ID, and it would show you, you know, you'd list out which test files changed in that commit.

Brian:
Okay.

Adam:
Or you can do with the dash dash stage. So if you've added some tests, but not others, you can just run the ones that you've added.

Brian:
Okay. Well, I'm going to ask a surprise question that we didn't plan. because one of the things I often am doing is, like we talked about, that I knew I changed something on a different branch. Can I diff what I'm currently looking at with a different branch?

Adam:
Yes, you can. You can diff any two commits. There's a syntax. It's with three dots to diff between two commits.

Brian:
Okay. Okay, and then it doesn't matter what branch it's on. It's just the two commits. Okay, got it.

Adam:
You can use a branch name or a commit ID or a tag name, like anything that points to a commit.

Brian:
Oh, I can use the three dot thing with just the branch name?

Adam:
Yeah.

Brian:
Ah, that's cool. Yeah.

Adam:
So, in fact, for this tip, you could run git diff main dot dot dot, and then with the dash dash name only start tests, and that will list all the test files you've changed on this branch since it branched off main.

Brian:
So much power. This is great. Yeah, neat. Cool. Okay. How about the third tip?

Adam:
Okay. This one's just the whole command, git bisect. Have you ever used bisect?

Brian:
I know of it, but I haven't. Tell me what it is.

Adam:
The way I think of it is that we use git log or git blame to look at the history of when a line of code changed. But when it comes to detecting actual behaviors, we need to run the code so that's where git bisect can step in, and the reason it's called bisect is because it applies the algorithm of a binary search it splits the history in half and then tests the middle and then goes did the thing change at this point, an example I wrote out in my book the first introduction is about a visual regression so you've got a website you've got some CSS but it's not clear why a footer has changed, you can't really tell it from looking at the code. Perhaps you're not like a CSS wizard. So instead, you decide to step through the history using bisect and iterate down to find the commit that changed the appearance in that way.

Brian:
Well, how would you, so you'd have to write it, does it run a function or something to tell you the answer? Like the question of when did the footer change or something?

Adam:
Yeah, so something like a visual regression, it's probably not worth your time to figure out how to write a script to like snapshot the website and tell what size the footer is. But you can give.

Brian:
It is, it is worth it for me. Okay.

Adam:
Yeah, maybe a chat jibbity can write it for you. Yeah. But yeah, if you want to do it fully automated, you can use git bisect run. And in that case, you tell it, you know, between this commit and this commit, something changed. You know, it could be something broke that didn't have a test. And I've written the test now. Or it could be, you know, something got slow. As long as you can write some small script that can detect that change. So like it might run your program and see, did it take under one second? Or did it take more than one second? and exit with an exit code appropriate to tell git bisect what happened. Then you start the bisect, you tell it here, run this tool, and then it goes do-do-do-do-do-do-do, and it can make you feel very powerful when it happens as it's like running your script over and over, narrowing down on that point in history where something changed.

Brian:
So could I give it like a pytest command line?

Adam:
Absolutely.

Brian:
Okay.

Adam:
The one key thing with pytest there or any testing tool is you don't want it to be something that's checked in because as it goes back through history, that checked-in test file might disappear. So you might write a new test file with just the one test.

Brian:
Oh, you can do it with it? Okay.

Adam:
And as long as it's not checked in, it doesn't collide with a name in your history, then as you go back, the file will stick around.

Brian:
Okay, but if we had at some point, this test is crashing now and we've had both code changes and test changes, I could use a committed one and it would run, run it. As long as it existed in both times, it'll find out when it failed. Okay. Maybe I didn't get the first part that you said. So I was assuming that it had to be an automated thing. We can use GitBisect without automation?

Adam:
Right. When you start it, it doesn't do anything automatically. It just starts the search process and it jumps back what's in time to that midpoint, that initial midpoint. And you can either do this GitBisect run to say here's my automated tool but if you don't then it's just waiting for you to run either like old or new commands you can say git bisect old this is showing the old behavior whether that's the footer is the correct size when it's changed to be big or whether a test is failing that has started passing unexpectedly like as long as you can divide like what the old behavior is at the start the new behavior and do something you can just tell git hey this commit it's showing the old behavior.

Brian:
Okay, so with git, the git bisect new and old, those are how you navigate through so if you say that, does it go to the next, pick a different one or something?

Adam:
Exactly. Then it just iterates on the search. If you're trying to change detect something from old to new, you're in the midpoint of the history and you say this is showing the old behavior, it's going to branch to the right and pick the midpoint between the middle and the history and the end. So like three quarters are long. And then it's just going to jump there and it's going to be waiting again for you to tell it.

Brian:
Okay. Oh, that's cool. Yeah. Especially, that might be the first place to start. So you don't have to automate something.

Adam:
Yep. Absolutely. And then even if you do have something automated with like pytest for like even a thousand commit history, you're only going to run like ten times. So running a command manually and then telling Git whether it passed or failed might be faster than figuring out how to automate it.

Brian:
Yeah, or let's say in an e-book sense. So if you're writing an e-book and you wanted to see when things got mucked up, you could have a... Either you could wait for it to stop and just run your build command manually or build the book as a command. I'd probably just do it manually. Run it, see it, and then you can determine old or new and keep going.

Adam:
Yeah, the process of jumping back in history is going to be very fast for most repositories.

Brian:
Yeah. Yeah, and actually, I remember once using this even... So at first guess, like with a test suite, you'd be like, wow, you'd only want to do this if you had a really fast test suite. But you kind of have to do that even if you... I mean, even if you don't have a fast test suite, this is going to be faster than manually trying to figure it out. Because you could just kick it off, get it started, and go to lunch, come back, see where it's at.

Adam:
Yeah. I know some people have bisect CPython itself. that's an overnight process if the if the bug is like very arbitrary i don't know where it could be.

Brian:
Well and also like let's say you're let's say you're not testing python maybe you're testing something that has a compile process attached to it it still might it still might be a good idea to to automate that just so that you don't have to sit around maybe it's maybe it's a weekend task but it's better than you working the weekend and trying to figure it out. You know, absolutely.

Adam:
Yeah.

Brian:
Get to it. Bicec, do it for you. These are great tips. So now to be, if people want to know more tips, then they, they'd head over. We'll leave a link in the, the show notes, but to, to, for people to pick this up themselves. And I definitely am going to look through this. I know there's a bunch of stuff that I want to let, Ooh, you have a chapter on log and ref log, and I've never really understood that. So I'm definitely going to have to read that. So that's cool.

Adam:
Yeah.

Brian:
Nice.

Adam:
Enjoy.

Brian:
Thanks, Adam, for showing up. One of the things we, since I think the last time we talked was many, many years ago. So I'm going to go ahead and ask because I can't remember. So tell me again a little bit about yourself.

Adam:
Yeah. So I'm Adam Johnson. I'm working a lot with Django. I'm a Django contributor and maintainer of a bunch of open source packages. I think I'm like 35 right now. I work as a Django consultant, so I'm doing a lot of short-term projects or part-time to help people keep their Django version upgraded, roll out the best ways to help their developers work faster, and some other things.

Brian:
Awesome. Great. And so we'll leave a link to the book, but also a link to how do people... Is that all right? Can we give a link to you to help people can get a hold of you?

Adam:
It's all on my website at adamj.eu.

Brian:
Okay. All right. Well, cool. Well, until next time, thanks a lot.

Adam:
Thank you, Brian.