LLMs Suck at Code
(Est. Reading Time: 2 min) So, here I am, tinkering with ChatGPT and other LLMs for over a year now, learning to leverage it for success for software engineering. It needs to be called out. LLMs suck at code. If a chat gets too long, they start to introduce artifacts and oddities into the code. For example, multiple unconditional return statements in a single function, adding a new one every time it "helps". It does have its good moments, though, as it helps discover bugs, sometimes. Ask it how to do something you've never done, and it will give you a basic understanding to work on. But it does not have the same logical mentality to actively participate in writing code. It gets worse. OpenAi researchers found that, beyond face value simple code, LLMs (even the unreleased frontier models) suck at code. Once the code starts to grow beyond a few classes, or a little complexity between different functions, it cannot hold the context in a way to provide meaningful output...