The main value I found from Copilot in vscode back when it first released was its ability to recognize and continue patterns in code (like in assets, or where you might have a bunch of similar but slightly different fields in a type that are all initialized mostly the same).
I don’t use it anymore though because I found the suggestions to be annoying and distracting most of the time and got tired of hitting escape. It also got in the way of standard intellisense when all I needed was to fill in a method name. It took my focus away from thinking about the code because it would generate plausible looking lines of code and my thinking would get pulled in that direction as a result.
With “agents” (whatever that term means these days), the article describes my feelings exactly. I spend the same amount of time verifying a solution as I would just creating the solution myself. The difference is I fully understand my own code, but I can’t reach that same understanding of generated code as fast because I didn’t think about writing it or how that code will solve my problem.
Also, asking an LLM about the generated code is about as reliable as you’d expect on average, and I need it to be 100% reliable (or extremely close) if I’m going to use it to explain anything to me at all.
Where I found these “agents” to be the most useful is expanding on documentation (markdown files and such). Create a first draft and ask it to clean it up. It still takes effort to review that it didn’t start BSing something, but as long as what it generates is small and it’s just editing an existing file, it’s usually not too bad.
Where I found these “agents” to be the most useful is expanding on documentation (markdown files and such). Create a first draft and ask it to clean it up.
How would the thing know what you were thinking and intending, what were the requirements, constraints, pre- and postconditions? How do you work around the risk that you end up with something that looks like useful information - but isn’t ?
Because you created a first draft. Your first draft should include all that info. It isn’t writing the whole doc for you lol, just making minor edits to turn it from notes into prose.
Without that? No clue, good luck. They can usually read source files to put something together, but that’s unreliable.
I find getting the LLM to either generate or rephrase documentation gives a distinctly worse result than doing it myself. I was in a hurry to document a new API from scratch recently and thought I’d try Copilot, but the results were overly verbose and sometimes inaccurate so I ended up rewriting all of it.
The LLM is best for boilerplate code that is easily predictable and verifiable. Beyond that it’s sometimes good for initial suggestions if you don’t know where to start with a tool, after which you can go to the actual documentation. But you’ll need to do that, because half the time the suggestions use nonexistent APIs and methods.
I have always thought that writing code is the easy part of being a developer. The hard parts are the parts management doesn’t appreciate: clarifying requirements, architecting new systems, translating business goals into something codable, letting egotistical know they’re not making sense without offending them, designing effective testing processes, persuading management to prioritize reducing technical debt, and integrating and maintaining existing systems. Maintenance is a huge part of the job that no one gives you credit for. Oh, and if you ever touch the front end, CSS.
I don’t use it anymore though because I found the suggestions to be annoying and distracting most of the time and got tired of hitting escape
Same. It took longer for me to parse and validate the suggestion as it did for me to just type what I wanted.
I do like the helper for more complex refractors.
Where you have a bunch of similar, but not exactly the same, changes to make.
Where a search & replace refactor isn’t enough.
It manages to figure out what you are doing, highlights the next instance of it and suggests the replacement.
I don’t think I’ve seen it make a mistake doing that, and it is a useful speedup.
I guess the LLM already has all the context: the needle, the haystack and the term.
The main value I found from Copilot in vscode back when it first released was its ability to recognize and continue patterns in code (like in assets, or where you might have a bunch of similar but slightly different fields in a type that are all initialized mostly the same).
I don’t use it anymore though because I found the suggestions to be annoying and distracting most of the time and got tired of hitting escape. It also got in the way of standard intellisense when all I needed was to fill in a method name. It took my focus away from thinking about the code because it would generate plausible looking lines of code and my thinking would get pulled in that direction as a result.
With “agents” (whatever that term means these days), the article describes my feelings exactly. I spend the same amount of time verifying a solution as I would just creating the solution myself. The difference is I fully understand my own code, but I can’t reach that same understanding of generated code as fast because I didn’t think about writing it or how that code will solve my problem.
Also, asking an LLM about the generated code is about as reliable as you’d expect on average, and I need it to be 100% reliable (or extremely close) if I’m going to use it to explain anything to me at all.
Where I found these “agents” to be the most useful is expanding on documentation (markdown files and such). Create a first draft and ask it to clean it up. It still takes effort to review that it didn’t start BSing something, but as long as what it generates is small and it’s just editing an existing file, it’s usually not too bad.
How would the thing know what you were thinking and intending, what were the requirements, constraints, pre- and postconditions? How do you work around the risk that you end up with something that looks like useful information - but isn’t ?
Because you created a first draft. Your first draft should include all that info. It isn’t writing the whole doc for you lol, just making minor edits to turn it from notes into prose.
Without that? No clue, good luck. They can usually read source files to put something together, but that’s unreliable.
I find getting the LLM to either generate or rephrase documentation gives a distinctly worse result than doing it myself. I was in a hurry to document a new API from scratch recently and thought I’d try Copilot, but the results were overly verbose and sometimes inaccurate so I ended up rewriting all of it.
The LLM is best for boilerplate code that is easily predictable and verifiable. Beyond that it’s sometimes good for initial suggestions if you don’t know where to start with a tool, after which you can go to the actual documentation. But you’ll need to do that, because half the time the suggestions use nonexistent APIs and methods.
I have always thought that writing code is the easy part of being a developer. The hard parts are the parts management doesn’t appreciate: clarifying requirements, architecting new systems, translating business goals into something codable, letting egotistical know they’re not making sense without offending them, designing effective testing processes, persuading management to prioritize reducing technical debt, and integrating and maintaining existing systems. Maintenance is a huge part of the job that no one gives you credit for. Oh, and if you ever touch the front end, CSS.
Same. It took longer for me to parse and validate the suggestion as it did for me to just type what I wanted.
I do like the helper for more complex refractors.
Where you have a bunch of similar, but not exactly the same, changes to make.
Where a search & replace refactor isn’t enough.
It manages to figure out what you are doing, highlights the next instance of it and suggests the replacement.
I don’t think I’ve seen it make a mistake doing that, and it is a useful speedup.
I guess the LLM already has all the context: the needle, the haystack and the term.