Needle 2.0.5: Pilot the 14 MB Tool-Calling Model Behind a Safety Gate

Needle 2.0.5 makes a narrow but important edge-AI proposition: turn natural-language commands into typed tool calls or structured data without sending the prompt to a cloud model. Cactus Compute packages the 45-million-parameter model and native inference engine as a 14 MB artifact, with Python, mobile, desktop, WebAssembly, and embedded targets.

That is compelling for a thermostat, wearable, home controller, or small robot. It is not a safe authorization layer. The Python package's convenient run() method executes the functions selected by the model immediately; it does not enforce the model's confidence score. The native library can also be fetched from a mutable Hugging Face repository when it is missing locally. Current release automation tests the Python layer without fetching that engine.

OpenSourceChoice verdict: pilot Needle for narrow, reversible, low-risk commands behind an explicit policy gate. Use complete() to inspect the proposed call, authorize it against an allowlist, require confirmation for consequential actions, and pin the native artifact offline. Do not use the default automatic execution path for locks, payments, medical devices, destructive operations, or privileged administration.

This is a researched technical analysis, not a hands-on benchmark. OpenSourceChoice reviewed the 2.0.5 tag and package, current source and commit activity, model repository, release workflow, tests, issues and pull requests, licenses, security posture, published architecture paper, and current alternatives. We did not run Needle on a device, reproduce Cactus Compute's speed or accuracy results, or fine-tune the model.

Executive verdict

Why this matters now

Needle 2 launched on August 10, and Cactus Compute published six PyPI versions from 2.0.0 through 2.0.5 in six days. Version 2.0.5 followed on August 15. The repository was still receiving commits that day, including engine-version and error-handling work. This is an active release train, not an old project rediscovered by a recent article.

Interest also appeared in more than one place. GitHub's daily Trending page listed cactus-compute/needle on August 16 and displayed 547 stars gained that day. The repository had approximately 6,100 total stars when observed. Separately, the August 10 Needle 2 launch discussion in r/LocalLLaMA attracted sustained comments about device use, reliability, fine-tuning, Home Assistant, and the limits of such a small specialized model. GitHub activity and an independent practitioner discussion are useful timing signals; neither is evidence of technical quality.

No equally strong project-specific signal appeared in the checked X, XDA, Product Hunt, Lobsters, or current Hacker News results. The editorial case is therefore not that Needle dominates every social channel. It is that a rapid new release, unusually constrained hardware target, current repository activity, and real operator questions create a timely adoption decision.

What Needle actually does

Needle is not a miniature general-purpose chatbot. Its useful output space is deliberately constrained. An application describes functions and typed parameters; the model selects a function and fills its arguments from a command. The same mechanism can return structured fields for extraction or classification.

The current design has four relevant layers:

• a 45-million-parameter Simple Attention Network trained for tool selection and structured output;
• a compact .cact model/engine bundle using the project's two-bit CQ2 format;
• a native runtime built for macOS, Windows, Linux, Android, iOS-family platforms, WebAssembly, and other constrained targets;
• a Python API that exposes run(), complete(), evaluation, and LoRA fine-tuning workflows.

The maintainers describe a schema-constrained byte grammar, retrieval of the five most relevant tools, and a 256-token sliding context window. Those choices explain both the small footprint and the boundary. Needle can map “set the bedroom thermostat to 21 degrees” to a declared function. It is not intended to explain a heating fault, search documentation, plan an ambiguous repair, or generate open-ended prose.

The project reports a single 14 MB binary, roughly 28 MB for a full session, and hundreds of decoded tokens per second on devices such as a Raspberry Pi 5 and inexpensive phones. Those are publisher measurements, not independent OpenSourceChoice results. A deployment decision should use end-to-end latency and battery or power draw on the exact target device, not the project's best published number.

The shortest API path is the dangerous one

The package presents run() as the convenient high-level interface. Source at the exact 2.0.5 commit shows that it asks the model for calls and then invokes the matching Python functions. It does not compare the returned confidence with an application threshold before execution.

That difference matters. A tool call can be syntactically perfect and operationally wrong:

• the user may not be authorized for the selected device;
• “turn it off” may resolve to the wrong room or asset;
• a numeric argument may be valid but outside a safe operating range;
• a stale command can arrive after the physical state changed;
• a repeated request can duplicate a purchase, unlock, deletion, or message;
• prompt content from an untrusted channel can select a privileged tool.

Confidence is only evidence about the model's prediction, not permission to act. The safer integration is complete(): inspect the proposed function, arguments, and confidence; apply identity, device state, rate, range, and action-class policies; then execute through a separate adapter. High-impact actions need a human confirmation that names the target and effect.

This separation also makes testing possible. Model evaluation can run without touching real devices, while tool adapters can be tested with deterministic inputs. Keep model output as untrusted data until the policy layer accepts it.

Confidence helps, but its contract is narrow

Needle's base model returns a learned confidence score intended to support local-versus-cloud escalation. That is useful only after calibration against the deployment's own commands, languages, accents, tool descriptions, and error costs.

The documentation notes two important limits. Fine-tuned weights do not currently return calibrated confidence, and non-English inputs can receive a zero score. A multilingual product cannot inherit an English threshold and call the result safe. A custom LoRA cannot rely on the base model's calibration contract either.

Measure confidence separately for every action class. A false positive on “open the garage door” is not equivalent to a false positive on “turn on the desk lamp.” Define precision, recall, abstention, escalation, and maximum unauthorized-action targets per class. Reject the pilot if the application cannot create representative negative examples, ambiguous commands, adversarial phrasing, and out-of-distribution inputs.

Small artifact, larger dependency and operations surface

The advertised 14 MB describes the model/runtime artifact, not the full Python development environment. cactus-needle 2.0.5 requires Python 3.9 or later and declares JAX, JAXlib, Flax, Optax, NumPy, SentencePiece, and Hugging Face Hub among its dependencies. The package wheel itself is small, but installation, fine-tuning, caches, and platform-specific runtime files make the real footprint larger.

That is not a contradiction: a product can embed the native artifact without shipping a Python training stack. It does mean teams should distinguish three budgets:

1. the production binary and model on each target device;
2. the application wrapper, tool adapters, and operating-system integration;
3. the development and fine-tuning environment used off-device.

Measure cold start, resident memory, storage, sustained power, thermal throttling, and latency with the complete application. Include the cost of maintaining builds across phone vendors, operating-system versions, CPU architectures, WebAssembly browsers, and embedded boards. A tiny model does not remove the device matrix.

Native artifact provenance needs a production policy

When the native library is absent, the Python package searches local and cache paths and can download the platform artifact from Cactus-Compute/needle2 on Hugging Face. The current fetch code names the repository and file but does not pin a repository revision. That makes onboarding easy, while leaving a fleet exposed to mutable upstream state.

The model repository was public and ungated when checked, with artifacts for the declared platforms. Its metadata identified Apache-2.0. Ease of download is not a substitute for provenance. A production build should:

• fetch an approved version during a controlled build, not at first device use;
• record the repository revision and cryptographic hash;
• scan and test every native binary for every platform;
• store the approved artifact in a controlled internal registry or release bundle;
• set NEEDLE_LIB_PATH to the reviewed library and use HF_HUB_OFFLINE=1 where supported;
• reject an unexpected hash and retain the previous version for rollback.

The .cact bundle is tied to an engine version, so model and native runtime should be promoted as one immutable release unit. A partial upgrade can be a compatibility incident even when both filenames look correct.

Release maturity and test evidence

The pace is encouraging but early. The repository was created in February 2026, had 263 commits and 13 visible contributors when reviewed, and continued receiving maintainer changes through August 15. Contribution volume was highly concentrated: the leading contributor accounted for 231 of the contributions returned by GitHub, while the next contributors were in single digits. That is a dependency to monitor, not proof that the project will disappear.

The project had 15 open issues and 10 open pull requests in the GitHub interface on August 16. Current work includes a proposal to stop base agents from silently answering when tuned weights are used and a pull request to correct package license metadata. A recent fix addressed multiple instances overwriting shared active state. An open issue asks the project to stop using pickle in parts of the model workflow. These are normal signs of active development, but they affect a production readiness judgment.

The release workflow uses Python 3.12 and runs pytest -q -m "not slow". The test configuration skips native-engine inference tests when the engine is not installed, and the release job does not fetch it. The reasonable inference is limited: the published workflow verifies the Python package layer but does not by itself demonstrate inference across the entire native platform matrix. Teams should not treat a green package release as device certification.

The Needle repository did not publish a SECURITY.md or a GitHub security policy when reviewed, and no public repository advisory was listed. Its underlying Cactus runtime repository does publish a security policy. Absence of a public advisory does not establish absence of vulnerabilities. Before a consequential deployment, obtain a private reporting route, expected support window, affected-version process, and software bill of materials for shipped binaries.

Privacy and data flow

Once the approved artifact is present, inference can run on the device without sending prompts to a model API. That is Needle's strongest adoption argument. Local inference can reduce disclosure, latency, connectivity dependence, and recurring inference cost.

The application still controls several outbound paths:

• a selected tool can call a network service or expose data through its side effect;
• the default setup can download native artifacts from Hugging Face;
• optional synthetic-data generation can send examples and tool schemas to OpenRouter;
• an optional upload step can publish fine-tuned artifacts to Hugging Face;
• application logs and analytics can capture commands, arguments, confidence, identity, or device state.

Map those flows explicitly. Keep sensitive examples out of external synthesis unless the organization has approved the processor, retention, region, and contract. Redact logs, separate evaluation identifiers from user identities, and never let a model-selected function bypass the application's normal network and authorization controls.

License clarity is part of the pilot

The Needle source repository's license file is MIT. The Needle 2 model/native artifact repository on Hugging Face identifies Apache-2.0. Those are permissive terms. However, the tagged Python package metadata still declares Apache-2.0 while the source repository uses MIT, and an open pull request proposes correcting that metadata.

The related cactus-compute/cactus runtime source uses a separate source-available license. It permits personal, educational, and noncommercial use, non-profits, and organizations below both specified funding and revenue thresholds; other commercial use requires a separate agreement. That license is not the same as the permissive terms presented by the Needle-facing repositories.

The evidence does not justify claiming that every Needle deployment owes a runtime fee. It does justify a clarification request. A commercial adopter—especially one above the Cactus thresholds—should obtain written confirmation of which license covers the shipped libneedle binary, its corresponding source, and any incorporated Cactus runtime components. Record the answer, artifact hashes, notices, and source-offer obligations before production. This is an engineering governance recommendation, not legal advice.

What Needle does not replace

Needle does not replace application code, a workflow engine, an access-control system, or a general language model. It is a probabilistic parser specialized for selecting declared functions and extracting values.

Use ordinary deterministic parsing when commands follow a stable grammar and mistakes are expensive. Use a larger local model when the task requires broader language coverage, reasoning, explanation, or recovery from novel situations. Use cloud escalation only when the data path and latency are acceptable. Many products need a layered design: deterministic rules for critical commands, Needle for narrow natural-language variation, and explicit refusal or escalation for everything else.

Alternatives to evaluate in parallel

Compare accepted outcomes, not model size alone. The best baseline may be a small deterministic classifier if it meets the command coverage and safety target.

A practical evaluation plan

1. Start with one reversible domain

Choose five to twenty low-risk tools such as querying a sensor, changing a display setting, or controlling a test lamp. Exclude locks, purchases, account changes, medication, machinery, and destructive administration from the first pilot.

2. Freeze the complete artifact set

Pin Needle 2.0.5, the exact native engine, model bundle, Python dependencies, tool schemas, and application adapter. Record hashes and licenses. Keep the prior set available for rollback.

3. Build a labeled command corpus

Include normal phrasing, synonyms, missing arguments, wrong rooms or users, impossible values, negation, corrections, multiple languages, background speech transcripts, copied untrusted text, and requests for unavailable tools. Label the correct call, arguments, refusal or escalation, and authorization result.

4. Evaluate proposals without side effects

Use complete(), never real tool execution, for the first evaluation. Measure exact function accuracy, argument accuracy, false-action rate, abstention, calibration, latency, memory, and power. Break results down by tool and risk class so an easy majority cannot hide a dangerous minority.

5. Add an independent policy gate

Check user identity, tool allowlist, target ownership, current state, argument ranges, replay protection, rate limits, and confidence. Require explicit confirmation for every effect above the low-risk tier. Make tool adapters idempotent where possible.

6. Canary on simulated and reversible targets

Run shadow mode first, then a small canary with audit logs and a visible kill switch. Test artifact loss, corrupt downloads, offline startup, model/engine mismatch, process restart, concurrent instances, timeout, repeated command, and rollback.

7. Approve or reject with measurable gates

Approve a narrow production scope only if:

• the unauthorized-action count is zero in the agreed representative corpus and canary;
• per-tool precision, argument accuracy, and abstention meet documented thresholds;
• confidence is calibrated for the deployed base model, language, and action class;
• p95 latency, memory, power, and cold-start fit the target device budget;
• every shipped native artifact is pinned, hashed, reviewed, and recoverable offline;
• every side effect is authorized, logged, bounded, and reversible where possible;
• the deterministic fallback and previous artifact restore service inside the rollback objective;
• licensing and security-reporting questions have written owners and answers.

Reject or pause the adoption if success depends on run() executing unchecked calls, the team cannot build a negative corpus, tuned weights remove the confidence behavior the design relies on, native artifacts arrive from mutable upstream state, or high-impact commands cannot require confirmation.

OpenSourceChoice conclusion

Needle 2.0.5 is interesting because it avoids pretending a tiny model can do everything. Function selection and structured extraction are narrow enough to fit devices that cannot host mainstream language models. Local execution can provide real privacy, offline, latency, and power advantages.

Its small size does not make its decisions safe. The default high-level execution path, mutable native download, young release process, concentrated maintenance, incomplete platform-test evidence, confidence limits, and license layering all move responsibility into the application team.

Pilot it as an untrusted command proposal engine. Pin the artifact, use complete(), authorize every proposed action outside the model, confirm consequential effects, and measure against deterministic and larger-model baselines. If those controls feel disproportionate, Needle is not ready for that product. If the device budget makes them worthwhile, the project offers a distinctive experiment—just not an autonomous control plane.

Official sources and further reading

• Needle repository — Cactus Compute; current README, source, MIT license, activity, issues, and pull requests; accessed August 16, 2026.
• Needle 2.0.5 tagged source — Cactus Compute; published package behavior and documentation at commit c152cc4d9821a002285f85aaf58876c6d60541fe; tagged August 15, 2026.
• cactus-needle 2.0.5 on PyPI — package metadata, Python requirement, dependencies, and artifacts; uploaded August 15, 2026.
• Needle 2 model and native artifacts — Cactus Compute; Apache-2.0 metadata and current platform files; accessed August 16, 2026.
• run() and complete() implementation — Cactus Compute; native-library resolution and direct Python tool execution in 2.0.5.
• Native artifact fetch implementation — Cactus Compute; Hugging Face repository, engine version, platform file selection, and download behavior in 2.0.5.
• Release workflow and test configuration — Cactus Compute; Python release checks and native-engine test skipping when the engine is unavailable.
• Needle MIT license — Cactus Compute; source redistribution and modification terms for the tagged repository.
• Open package-license correction PR #68 — upstream metadata correction proposal; accessed August 16, 2026.
• Open tuned-weight behavior PR #71 — upstream proposal concerning base-agent answers with tuned weights; accessed August 16, 2026.
• Open pickle issue #36 — upstream model-workflow concern; accessed August 16, 2026.
• Cactus runtime repository and current license — Cactus Compute; related native runtime source, commercial thresholds, releases, and current maintenance; accessed August 16, 2026.
• Cactus security policy — Cactus Compute; reporting and supported-version information for the related runtime; accessed August 16, 2026.
• A Controlled Study of Attention-Only Transformers — Cactus Compute authors; submitted July 20, 2026; project-authored architecture study, not independent validation of Needle 2 device safety or performance.
• FunctionGemma model card — Google; small function-calling comparison candidate and usage constraints; accessed August 16, 2026.
• GitHub daily Trending — discovery signal observed August 16, 2026; listed Needle with 547 stars gained that day, not treated as proof of quality.
• r/LocalLLaMA Needle 2 launch discussion — community-interest and practitioner-question signal; posted August 10, 2026; not used as primary technical evidence.