Although it may seem counterintuitive, AI tools can be of greatest help to those who already know what they’re doing.
Three engineers walk into a bar. Together, they have 25 years of experience, but 15 of those belong to one of them. How many years of experience do the other two engineers have?
This seems like a typical question people are asking ChatGPT these days. After all, outsmarting AI is the ultimate goal – or is it?
We asked three flesh-and-blood engineers about their experience of working with AI coding assistants GitHub Copilot and Chat GPT, and found out whether they recommend using them – and in what cases.
These are their stories. *tn tn*
The flesh-and-blood engineers who participated in this experiment
Marin Teskera, a junior developer fresh off of JavaScript course at Infinum Academy. He is at the beginning of his career and eager to learn the intricacies of programming.
Tomislav Habalija, a JavaScript engineer turned JavaScript team lead, who also wrote the Angular course curriculum for Infinum Academy and mentored Marin at the JavaScript course. He’s been in the biz for seven years.
Darko Kukovec, an experienced JavaScript and TypeScript engineer with +15 years of code writing under his belt. He is the head of frontend development.
Where did AI code writing assistants prove to be useful?
Using a coding assistant for writing boilerplate code
Nobody likes it, but boilerplate is always there. It’s a hassle to read and maintain it, so any kind of a helping hand is a godsend. Whether it’s lists of data or some repetitive code, types, or just long names of variables and functions, it’s easier to use an AI coding assistant to write it, and then check if everything is ok than to do the whole thing from scratch.
Also, as our Marin puts it, “doesn’t it feel great when the AI coding assistant recommends something you were going to write anyway?” #sanityCheck
Using a coding assistant for writing documentation
Sometimes you know exactly what you want to do, but you can’t fully remember the documentation for some API (e.g. DOM, or even your own code). The copilot might be able to autocomplete big chunks of the code without having to switch tabs or apps to check the docs.
What tasks did AI code writing assistants struggle with?
Using AI tools for learning something new
With a lot of JavaScript tooling being written in Rust nowadays, Darko Kukovac decided to try out first-hand what all the fuss was about. He opted for Rust when solving the 2022 edition of advent of code.
“In preparation, I had just enough time to read the first six chapters of the Rust docs – not great, not terrible. This was enough to get familiar with the basic concepts, but without any practical knowledge, it would surely be quite annoying to google solutions for each trivial issue I encountered. After all, that’s what happened when I tried Kotlin a few years ago,” Darko said.
Copilot did its job and got Darko through the first few days with much less frustration than expected. Of course, he still had to google things, but mostly on the conceptual level and not syntax or trivial issues. After a few days, he noticed his understanding of the language got better, and he was able to better understand Copilot’s suggestions and assess if they were good solutions or not.
Looking back at the code that Darko wrote himself and the code where he trusted Copilot (almost) blindly, he concludes:
“The Copilot code is somewhat weaker because it adds extra steps or does some things in a less-than-ideal way. However, I do believe it can help engineers kickstart their learning experience, which is a plus.”
Using code you don’t understand
What happens when the AI helper recommends code you don’t understand?
By just mindlessly pressing the tab key, developers who are still learning the ropes can miss the opportunity to learn by trial and error.
For example, in last year’s edition of Infinum Academy, our mentor Tomislav Habalija had a few students who were writing suspiciously clean code:
“I had a hunch something was going on until I realized they were using GitHub Copilot.”
It is a tool, or as they say on their website – AI pair programmer, which speeds up your developing process by giving you suggestions on what to write next.
For example, let’s say you want to write a simple function that calculates average numbers. Just by writing the function calculateAverage and selecting the first GitHub Copilot example, you will get the correct implementation.
function calculateAverage(numbers) {
const sum = numbers.reduce((acc, number) => acc + number, 0);
return sum / numbers.length;
}
This can be a great time saver for a senior developer who knows what they are doing but a big pitfall for junior developers. “By writing code from scratch, you are learning, and while using tools like this, you are just writing code,” says Tomislav.
Introducing bugs while you introduce “correct” code
If you’re not that experienced, blindly trusting the coding assistant can result in using code that is plain wrong.
But even if the code is correct and it works, it doesn’t always follow best practices. It might also introduce errors that a beginner doesn’t know how to fix. That is really not ideal when working on a bigger project because when another programmer starts to work on the code, they may have a hard time following it.
Case in point: Tomislav Habalija tested this when he was writing a regular expression:
“I had to write a regex for removing query params from the string. Easy enough, right? So I asked the AI.”
This was Tomislav’s prompt:
Write a regex for removing query params from the string using javascript.
And he received a correct response:
const removeQueryParams = (url) => url.replace(/\?.*/, '');
This solved one case but added a bug to the rest of the code because the team was using templated links. ChatGPT couldn’t know that, as Tomislav did not mention it.
Back to the drawing board, Tomislav tried another prompt:
Write a regex for removing query params from the url that contains templated strings. Templated part starts with { and ends with }. Templated part must not be removed from the string.
This time, the answer was slightly better.
const removeQueryParams = (url) => url.replace(/\?(?:[^{]*&)?([^#]*)#?.*/, '$1');
“Regex is bigger, it should be correct, right? Right? I have tried it out in my code, and I was surprised when the tests started failing. The answer was still wrong,” Tomislav decided before giving AI one final chance.
The provided answer still removes the templated part. For example, regex should not remove “{?foo}” or “{?bar}” from “http://www.test.com{?foo}/bar{?bar}?first_param=1&second=2”. Can you try again?
AI’s response was “Here is a modified regular expression that will remove the query parameters from a URL string, while leaving any templated parts of the URL intact:”
url.replace(/\?(?:[^?{]*&)?([^#]*)(#?.*)/, '$1$2');
This answer was even worse than the one before. The answer Tomislav was looking for was:
url.replace(/([^{])\?[^/]+/, '$1');
And he couldn’t get AI to come up with that. At least not with the questions and descriptions he provided.
Privacy considerations
Since both GitHub Copilot and ChatGPT use their servers to generate results, the question is what happens with the data entered. How is it used? This might be a very important question if you’re working on a project that is not open source.
In the case of Copilot, there’s an option in settings that allows it to use your code to train the AI – this doesn’t mean that someone will get your snippet of code, but someone might get a code snippet that was in part inspired by your code. If you’re working on code that is not open source or the license doesn’t allow this type of usage, you should disable this option.
On the other hand, ChatGPT doesn’t have such settings, and all entered data could be used to train the AI. That’s why you should always be careful what you enter into the system and obfuscate any information you don’t want to share with anyone.
Turns out, AI coding assistants are only as smart as the engineer using them
Our engineers all agreed that using an AI coding assistant makes the most sense for more experienced developers who know what they are doing.
For people trying to learn, like our junior engineer Marin, using a tool like this brings the same benefits as for the experienced developer. However, they should be careful about using code they don’t understand because it can affect their learning curve, or they might even end up introducing the wrong functionalities.
Using AI coding assistants with a grain of salt can speed up development, but for now at least, it is not ready to take over the wheel.
