I spent way too long trying to find and configure a local AI coding mentor I could actually trust

2026-08-14

A little background

I'm a hobbyist, although I like to think I'm slowly becoming a bit more than that. If I'm honest, though, I'm still pretty deep in what everyone calls tutorial hell. Whatever I happen to be building, I still often feel like a beginner underneath it all.

That hasn't stopped me from starting a lot of small personal projects.
I get a new idea roughly every 15 minutes, and with AI doing some of the heavy lifting, I've been able to build things much faster than I ever could have the traditional way.

There was a cost to that, of course. I ended up with a pile of half-finished projects, even more ideas waiting behind them, and a growing list of subscriptions and API costs.

Recently I wanted to cut back on the subscriptions. At the same time, I promised myself I'd stop defaulting to vibe-coding and get back to learning by actually doing the work. It turned out that building things on my own felt genuinely difficult after relying on AI for so much.

Meanwhile, I had 16GB of VRAM sitting in my GPU doing basically nothing most of the time. So I tried a different approach: cut most of the subscriptions and run AI locally on my own hardware, almost for free.

The real question was whether that would actually work. Could I run a model that was good enough to act as a mentor? I wanted something that would nudge me towards an answer instead of immediately handing me the code, and something I could trust when it talked about my own project.

The short answer is: sort of. What was supposed to be an afternoon of testing somehow turned into a week-long investigation that ate a fair chunk of my evenings. In the end, I learned less about which model was the smartest and more about how to catch one confidently making things up. The same problem kept showing up in different forms the further I went.

This is the writeup of that process, from start to finish. It isn't really a benchmark. It's closer to a record of what I tried, what failed, and what I ended up trusting.

The setup

The setup was nothing fancy, at least at first. I installed Ollama on my machine with an RTX 5060 Ti and 16GB of VRAM, then used Pynventory, a small Python inventory app I've been working on, as the test project. I connected the models to VS Code, PyCharm, and a couple of terminal coding agents so I could see how they behaved in different environments instead of judging them from a single chat window. For a rough speed comparison, I used ollama run <modelname> --verbose.

The methodology

This was the part that surprised me most. A model could tell me something completely wrong about my own code with exactly the same confidence it had when it was right. Unless I checked carefully, the difference wasn't obvious. Eventually I stopped asking myself whether an answer sounded smart and built most of the testing around one simple, fairly boring prompt:

"Read db_operations.py, then explain how to test the functions in it."

If I asked about my code without explicitly telling the model to read the file first, there was a decent chance it would invent something plausible and carry on as if it had seen the code. A smarter model didn't necessarily fix that. What helped was making it use the file-reading tool every single time.

The other habit I picked up was to never trust one answer on its own. I ran the same question through several models and compared what they found. If three models independently pointed to the same bug, there was probably something worth investigating. If only one mentioned it, I went back to the file before believing anything.

Initial testing results

By the end of the first day, I was mostly rotating between gpt-oss:20b, qwen3.6:35b-a3b, and a lesser-known model called laguna-xs-2.1. A handful of others came and went along the way.

The phantom feature. Three models, in three separate sessions, told me to "add a search function" to the app. The app already had one. It just wasn't listed in the README's command table, so any model that skimmed the README instead of reading the code confidently suggested that I build something that already existed.

The typo that turned into a permission request. gpt-oss:20b kept calling my project "pyninventory" instead of "pynventory." It did this across three different tools on two separate days. At one point it even requested file permissions for a path with the wrong project name in it.

The self-contradiction. At one point, laguna-xs-2.1 gave me a genuinely good and honest answer about what it couldn't confirm. Two messages later, it flatly told me to "implement the search function" as if it didn't exist. Same model, same session.

The best single answer of round one came from qwen3.6:35b-a3b. I asked how the app would hold up with 50 concurrent users, expecting a generic warning about SQLite. Instead, it traced through the code and noticed that my stock-update logic performs a blind overwrite instead of an atomic decrement. That is a real and fairly subtle race condition, and finding it required more than repeating general advice about database concurrency.

After round one, I settled on laguna-xs-2.1 as my daily default. It was fast, and it had made the fewest claims that didn't match the files. I kept qwen3.6 around for harder problems.

Comparing to frontier models

After getting frustrated with the first round, I tried GPT-5.6-Luna (med) through Codex. I'd kept my ChatGPT subscription, and I wanted to know whether a stronger coding tool with access to the whole repository would avoid the same problem. For the most part, it did.

I gave it instructions for the kind of mentor I actually wanted. It should read the relevant files before answering, explain what it found, ask when something was unclear, point out bugs, and prefer hints or small steps over handing me a finished solution straight away. With those instructions in place, it was much easier to use AI as a learning tool without slipping back into vibe-coding.

The almost perfect local model, until it wasn't

Partway through the testing, because apparently I hadn't tried enough models already, I found Mellum2. It's a small model from JetBrains with 12 billion parameters, although it only activates 2.5 billion for each token. In theory, that should make it fast. In practice, it was easily the quickest local model I tested all week, with sub-second load times and more than twice laguna's token rate.

The first proper test was rough. I asked it how I should test db_operations.py, and it returned a database schema with the wrong column types. It listed weight and in_stock as INTEGER, even though they are REAL in the actual file. The strange part was that it really had read the file. It just replaced what it saw with a more generic, plausible-looking schema. Those types also happened to relate to a bug I already knew about in the project, which made the mistake even less helpful.

I checked how JetBrains describes the model, and they don't really present it as a general-purpose assistant. They call it a "focal model," built for narrow, fast tasks inside a larger pipeline rather than broad reasoning. Their own benchmarks also show it falling behind similarly sized models on general reasoning. That seemed to explain what I'd just seen, so I was ready to write it off.

Then I remembered something I'd come across earlier: reasoning mode can sometimes make hallucinations worse when the main job is to stay faithful to a source. Reading a file and explaining what is actually in it is exactly that kind of task. Mellum2 comes in two variants, one with reasoning and one without, and I'd been forcing reasoning onto the non-reasoning "Instruct" checkpoint. In hindsight, that wasn't a sensible setup.

I turned reasoning off and ran the schema test again. This time it got everything right. I repeated another test where it had previously failed to ask a useful clarifying question, and that worked too. Instead of vaguely asking, "what kind of search function is this?", it asked which file contained the function. Then I gave it the same phantom-feature trap that had fooled three models in round one. It handled that better than anything else I'd tested all week, correctly noticing that the search function already existed and only needed better filtering.

After three clean results in a row, I was ready to add it to the lineup.

Then I let it try an actual file edit. It "optimized" the SQL in my search function by silently dropping an entire column from the query. item_type was simply gone, and the explanation never mentioned it. The edit was presented as a tidy little performance improvement. I rejected it, thankfully. If I'd skimmed the diff instead of reading it properly, I could easily have accepted a real regression because the explanation sounded reasonable.

That failure stuck with me more than the others. A model can repeatedly earn your trust while reading and explaining code, but that doesn't automatically make it safe to edit files. It doesn't always fail with a crash or an obvious error. Sometimes it makes a bad change and then gives you a perfectly calm, plausible explanation of something it didn't actually do.

Three more, three different kinds of no

Once I had a testing method that seemed useful, I ran a few more models through it. None of them made the final setup, but they each gave me a different reason why.

qwen3-coder:30b actually did the job well. It corrected itself after guessing the wrong file path and eventually gave me a complete, accurate answer. The problem was simply that it didn't fit my graphics card. At 19GB, part of it had to run on the CPU, making it much slower than models around half its size.

qwen2.5-coder:14b never even tried to read the file. I tested it twice in two different environments, once through the VS Code extension and once through Codex, with the same result. It didn't read the file incorrectly. It made no attempt at all, even though the prompt explicitly told it to.

gemma4:e4b does support proper reasoning, and Google presents thinking mode as a major reason for its improved benchmark scores. The problem is that the toggle uses a special token in the system prompt rather than a normal setting that generic coding tools can easily change. It also couldn't recover from a wrong file path the way Mellum2 and qwen3-coder could. It just stopped and asked me to provide the right one.

The file path issue became a small investigation of its own. These three models, plus Mellum2 and gpt-oss earlier, all guessed the exact same wrong path at some point. The path led to the VS Code extension's own installation folder, not my project. I assumed the extension was causing it, so I ran the models again through the Codex harness after a bit of tinkering. The issue still didn't disappear completely. Two models failed to correct themselves even in the cleaner environment. I still didn't know exactly where the bad path came from, but I did learn which models would check their work after the first attempt failed and which ones would just give up.

Deeper down the rabbit hole

Somewhere along the way, I discovered that OpenCode has a free model gateway called Zen. DeepSeek V4 Flash was available through it at no cost, so naturally I added one more model to the test.

For the final comparison, I asked Codex and the free DeepSeek model the same open-ended question: as a total beginner, what should my next steps be in this codebase? Both gave good, well-structured answers based on the actual project. Neither one invented anything.

Codex found two extra things, though. First, it noticed that the help text tells users to type "weigh" for the weight field, while the code checks for "weight." That was a real bug I hadn't spotted myself. It also searched the rest of the codebase to confirm that a separate helper function was dead code before bringing it up. DeepSeek's answer was still solid, but it didn't catch either of those.

It still felt a bit strange to land there after spending so much time testing local models, but the comparison made my priorities clearer. Codex wasn't useful because it happened to give one impressive answer. It gave me a repeatable way to work across the whole repository, and I could keep my mentor instructions active globally.

What I chose

In the end, I kept two tools for two different reasons.

I gave everything else, including qwen3.6, gpt-oss, Mellum2, qwen3-coder, qwen2.5-coder, and gemma4, a serious try. None made the final cut. They weren't necessarily bad models, but each one filled a role that one of the two tools above already covered better for my setup.

Conclusion

I started all of this worried that I wasn't qualified to judge whether an AI's answer about my code was actually any good.

What surprised me was that finding a model I trusted didn't solve the problem for good. Mellum2 earned that trust while reading code and lost it as soon as I let it edit a file. Codex gave me a much better workflow, but I still had to check its work. Picking the right model matters, but staying willing to verify things matters more.

I didn't need to be an expert to catch most of these problems. I only had to notice when a model contradicted itself, repeated the same typo, or quietly removed a column during a supposed performance improvement. Most of the time, careful attention mattered more than deep technical knowledge.

This probably isn't where I'll stay forever either. I'll find another model, change a setting, or spend another week testing something I expected to take one afternoon. For now, though, the method is simple: ask more than one model, make sure they actually read the file, and keep checking their work even after they've earned some trust. Nothing fancy, but it worked for me. If you're experimenting with local models too, it might be worth a few of your evenings.

Update

A few days later, I ended up keeping my ChatGPT subscription and using Codex with the same mentor-not-solutions instructions as my global defaults. Laguna is still what I reach for first when I only need something basic. Codex is there when I need more depth or context from across the repository. Together, they come much closer to the coding mentor setup I was looking for in the first place.