Choosing an agent harness for my DGX Spark

I wanted my local models to take on more of my everyday work: fixing code, looking things up, reading documents and working with files. I also wanted to close the browser, get on with something else and come back to find the work still running or the results ready.

The models were already running on my DGX Spark. The next question was which agent harness to use with them. That’s the software that gives the model tools, keeps track of the task and works out the next step.

That brought me to Agent Zero, OpenHands, OpenCode, Cline, Aider, Pi, Hermes, LangGraph with Deep Agents, LM Studio and Unsloth Studio. Model studios and agent harnesses handle different parts of the setup. I’m comparing them against the same requirements to understand where each fits. None of them fully fits my needs yet. The main gaps for me are multi-day work, context management for smaller models, configurable RAG and control over inference settings.

What I need the harness to take care of#

Getting a response from a local model is a start. Once I ask it to edit a file or run a command, I need to know a few more things. Which files can it access? Is the command running on my machine or inside a container? Can I see what it did? If it stops halfway, can I pick up from there?

I look at the setup in four parts. The model server runs inference. The harness prepares context and calls tools. The task layer keeps track of progress and handles interruptions. The browser or terminal lets me follow the work and guide it. Even when one product handles all four, it helps to know what each part is doing.

So when a tool says it supports local models, I look a little closer. Can it connect to my endpoint? Can the model follow its tool format, use the context properly and recover when a step fails? All of these matter in day-to-day use.

Along with context and tool compatibility, these are the practical things I look for:

• Remote access: A stable browser or API address, such as a Cloudflare Tunnel hostname, with authentication and a way to reconnect to the work.
• Host and hardware access: Access to the GPU, files, installed environments and cached models. I check what runs directly on the machine, inside a container or through a host connector.
• Multi-day jobs: Work that continues over several days, with saved progress, checkpoints and a way to resume after interruptions. I need to return to a clear task state, including what is done and what is pending.
• Context management: A usable context budget for smaller models, with the task plan, decisions and relevant files carried forward as the job grows. Summaries and retrieval need to preserve the details needed for the next step.
• RAG support and configuration: Control over which documents are indexed, how they are split, the embedding model and what gets retrieved. I want to see the sources behind retrieved information and tune what enters the context.
• Model management: Finding, downloading and importing models from Hugging Face, keeping track of exact revisions and reusing the cache. Versioned downloads help me get the same model files again.
• Configuration: Controls for context, reasoning, quantisation, sampling and output limits, wherever the model supports them. I want to save these settings separately for each model and role, and have them applied consistently when switching models or resuming a task.
• Runtime control: Task state, queues, delegation and parallel workers, plus ways to guide or cancel work while it is running.
• Observability: A clear view of tool calls, errors, changes and task results, so I can follow what happened.
• Maintenance: The effort to set things up, update dependencies and keep the runtime working with the model backend.

The options I’m considering#

These tools serve different needs. Some focus on editing a code repository. Others work more like a personal assistant, or provide the pieces to build one. This is how I compare them against what I need.

Agent Zero brings projects, memory and delegated tasks into one workspace. OpenHands lets me define where software tasks run, and its SDK covers saving and restoring conversations. Its local-model guide also explains that some features depend on the model. For either tool, I’d check how these capabilities fit my workflow, hardware and recovery needs.

For coding, I’d start with how I prefer to work. OpenCode has a server API with asynchronous prompts and local-provider configuration. Cline keeps the work inside the editor and offers a compact prompt for local models. Aider uses a repository map for context, and its architect/editor modes separate planning from editing. It also works with Ollama. Giving each tool the same small fixes makes it easier to compare the edits and see how often I need to step in.

Pi and LangGraph are options when I want to build the workflow myself. Pi includes a durable runtime along with its agent and provider packages; permissions and sandboxing need to be set up as part of the deployment. LangGraph provides persistent state, while Deep Agents adds support for context, files and delegation. Both involve integration work on my side. Hermes covers recurring assistant tasks, with memory and skills to carry useful context from one task to the next.

A brief setup example#

In one setup, we ran Agent Zero in Docker with separate local inference services: a quantised 27B main model, a 9B utility model and CPU embeddings. A host connector gave it access to existing files, Python environments and GPU commands, using the host user’s permissions. A named Cloudflare Tunnel gave us a stable HTTPS address. It’s a simple example of how the workspace, inference, access to the machine and remote interface fit together.

One address to reach the setup#

I set up a named Cloudflare Tunnel to reach the workspace through a permanent HTTPS address. The hostname stayed the same across restarts, and the workspace remained behind its login. I could open the same address from another device without setting up a new URL each time.

The agent and inference services ran on the Spark, so closing the browser did not stop them. That was useful for longer tasks and for switching between devices. The Spark still had to stay online; the browser was simply how I accessed it.

Connecting the workspace to the machine#

The workspace ran in Docker, so its usual terminal worked inside the container. My existing Python environments, cached models and GPU tools were on the host. I connected those through a host connector, which gave the agent access under my normal user permissions.

I checked the connection with identity and GPU queries on the Spark. This made the distinction clear: the container tools worked in their own environment, while the host tools worked on the machine where my files and models already lived.

The models I tried#

I tried these models across the main agent setup and my own harness work. Each had a different role, and the prompt and tool setup made a difference to the result.

The 27B and 9B models became separate main and utility services. The 3B runs showed how much the prompt and tool interface could change the result, even with the same model weights. That made the model–harness combination an important part of my comparison.

The settings behind each model#

I saved context, reasoning, sampling and output limits separately for each model. The llama.cpp server handled inference, while the harness presets selected the model for each role. For the 27B main model and 9B utility model, I used:

• Context: A 128k-token limit for each model, with the harness’s history ratio set to 65% to leave room for instructions, tools and generation.
• Reasoning: Medium reasoning for the main model; reasoning disabled for the utility model used for extraction and summarisation.
• Output limits: 16k tokens for the main model and 8k for the utility model. The main model’s limit included its reasoning output.
• Requests: One inference slot per model, with requests to each service queued. Both services still shared the Spark’s GPU.

These choices also shaped the comparison. Alongside downloading and running a model, I looked at how easily each tool let me save its settings and load them again. Support for per-model load settings was one example of that.

Why I’m still looking#

The setup got my models, tools and remote access working together. But none of the options in this comparison has given me the complete workflow I want yet. These are the gaps that still matter to me:

• Keeping a job going for days: I need the task plan, completed steps, pending work and outputs to survive interruptions. When I come back, I should be able to see where the job stands and resume it without rebuilding the task from the chat.
• Keeping smaller models on track: A multi-day job can produce far more history than the model can use at once. Even the 128k context configured in my setup is a finite budget shared by instructions, tools, retrieved material and responses. I need the harness to carry forward the right decisions and evidence, and give the model a focused next step.
• Making RAG part of the workflow: I want to choose the documents, chunking, embedding model and retrieval settings. The agent should be able to fetch relevant material as it works, show where it came from and keep unnecessary content out of the prompt.
• Managing inference settings properly: I configured the main and utility models separately. I still want those profiles to fit neatly into the agent workflow: context, reasoning, sampling and output limits, alongside the quantisation and load settings supported by the inference backend. Switching models or resuming work should retain the intended configuration.

For now, I’m keeping these tools in the comparison. Each covers parts of what I need, but I haven’t settled on one for multi-day work with local models. I’m still looking for a setup that brings task continuity, context, retrieval and model configuration together.

AgentsSmall modelsDGX SparkHarnesses