Why I rebuilt ConsoleX

My first product was EvalsOne. It was built for teams making products on top of language models, and it answered two questions: are the model's answers any good, and did the last change make them better or worse?

I still think those were the right questions. But I was early. Few teams had AI products far enough along to need them answered systematically, and I never found a path to a business. I ran EvalsOne for a while, never landed a single customer, and shut it down.

So I put everything into ConsoleX. It began as a workbench for chatting with many models in one place. Over the next year I kept building: tool use and MCP, then agents, scheduled tasks and Skills. ConsoleX grew from a chat app into a place where AI could use tools and keep working on its own.

Then the ground moved. Models got far more capable, competition got fiercer, and the companies making the models started building the same things into their own assistants. Much of what I had added to ConsoleX was becoming a standard feature of apps people already use. That isn't a race a small team should run.

So I went back to the question I started with. Getting AI to do the work has become easy almost everywhere. Knowing whether the result is right hasn't, and neither has making the next run better. It took me two products to see where that matters most: in the work people do again and again.

That's what the new ConsoleX is for.

The second time

Most of the work people bring to AI isn't new. It's the recap after every client call, the article that has to become a post for each channel, the monthly report, the draft that has to follow the house rules.

You've probably had a chat where, after a couple of corrections, the result was exactly right. Then the same work came back the following week. You pasted the prompt again, repeated the same corrections, and read every line to make sure nothing had slipped.

I rebuilt ConsoleX for that second time.

ConsoleX is now an AI studio for work you have to deliver. Get it right once, in a chat or from a template, and save it as a task. Every run after that is checked against your standard. When a run misses, the miss can become a change to the task, and you decide whether it goes in.

Where a chat leaves off

A chat ends with an answer. For a question, that's the end of the job. For work you have to hand over, it's where your part begins:

• Next time, you explain it all again, including last week's corrections.
• Before you send it, you check it yourself, line by line.
• What can't be verified sits mixed in with everything else, for you to spot.
• When it misses, you work out what to change and rewrite the prompt yourself.

ConsoleX picks up from there.

How it works

1. Get it right once. Start from a template close to work you already repeat, or chat it through with your own material. Correct it until it's something you'd hand over.

2. Save it as a task. AI organizes the conversation into instructions that keep your corrections, and suggests checks for the result. You review both before saving. Templates come with their checks already set up, and you can change any of them.

3. Run it again. When new material comes in, run the task with it. Once it works the way you want, you can put it on a schedule.

4. Read the check report. Each check comes back passed, failed or needs review, with the evidence behind the verdict. The run's usage and estimated cost are in the same report, with the cost of checking listed separately. Then you decide: accept the result, ask for edits, reject it, or mark a verdict as wrong.

5. Improve the task, not just this result. A failed check, your feedback or your own request can become a suggested change to the task's instructions, with every edit marked. Nothing changes until you approve it. Each change you approve is a new version, and any earlier version can be restored.

6. See whether it helped. The next run uses the new version and is checked against the same standard. If the change didn't help, adjust it again or go back to an earlier version.

A real example

Here's a small task I ran while preparing this post. It drafts a short status email to a client about a website refresh: progress so far, the materials still missing, and a decision the client needs to make. The client and project are made up; the runs, checks and suggestion are real.

I got the email right in a chat and saved it as a task. AI turned the conversation into instructions and suggested checks: a subject line that names the project, under 100 words, end by asking the client to confirm their decision, and so on.

Then I added a check of my own: progress, missing materials and the decision should be three labeled bullet points. The next run failed it. The report says why, and shows the output it based the verdict on.

From that run, ConsoleX suggested a fix. My instructions had never asked for bullets, so it proposed adding that requirement while keeping the decision last, so the email would still end with the ask. I reviewed the change and accepted it.

The next run passed all five checks.

One passing run doesn't prove much on its own. That's why every run stays on record: you can see whether a change keeps holding, and go back if it doesn't.

What a check is, and what it isn't

• Rules first. Things that can be verified directly, such as required sections, links, length or banned words, are checked without a model.
• Evidence for judgment. Where judgment is needed, the verdict has to cite evidence from the run. If the evidence isn't there, the item is marked needs review instead of passed.
• Passed isn't a guarantee. A check compares the result with your standard. It doesn't promise that every fact is correct, and you can overrule any verdict.
• Files aren't opened yet. Checks read the final response and what the tools returned. They don't open generated Word, Excel or PDF files, so review those yourself.

Where to start

Four templates cover work that keeps coming back:

• Edit to your rules. Revise a draft to your rules and the latest feedback. Your facts stay as they are.
• Deliverable from your materials. Turn notes, transcripts or documents into a recap, brief or proposal, using only what they say.
• Content repurposing. Turn one article, video or episode into a post for each of your channels.
• Client report. SEO and AI-visibility numbers for a client's site, what changed, and next steps.

Each template brings its instructions, the tools it needs and its checks, and you can change anything before you save. Choosing a template doesn't run anything. Templates that write emails or posts only draft them: you review the result and send or publish it yourself. You'll find more in the template library.

Who it's for

I built ConsoleX for people who deliver the same kind of work again and again and answer for it themselves: freelancers, creators, consultants and founders running a one-person business. If you only ask AI the occasional question, a chat assistant is enough.

What hasn't changed

If you've used ConsoleX before, chat is still here, with leading models, your own API keys, tools, Skills and Agents. Your chats are where you left them. What's new is what happens after the chat.

Plans and costs

Plans are simpler now. The 3-hour and weekly limits are gone; each plan has one monthly allowance instead, and runs and checks use it first. The Free plan includes 3,000 Credits a month. See pricing for the other plans.

Every run records the model, usage and estimated cost, and checks are estimated separately. Estimates can be incomplete, don't include charges from external services, and aren't a spending cap.

Try it on one piece of work

Pick the thing you keep redoing from scratch. Start from a template, adjust it until it's right, and save it as a task. The next time that work comes up, run it and read the check report.

I've been early before. This time I'd like to hear from you early: tell me what you'd want checked, and what got in the way. Write to contact@evalsone.com. I read everything.

— Quentin