A framework stack is a deployment risk.
Most local-inference setups sit on a Python environment, a deep-learning framework, a numeric library and a toolkit. Any one of them can change or break the run on a locked-down machine.
VEIZIK for business
Run supported models inside your environment.
Evaluate local inference with explicit runtime dependencies, model residency and startup behavior on your own hardware.
You keep the weights, the hardware and the data path. veizik is a local runtime, not a hosted API.
Operating problem
What an on-premise review has to settle.
Three questions come up before a model can run inside your environment.
The data path should be written down.
Review needs to know what runs, what connects and what is downloaded — separately for inference, activation, downloads and analytics — not one blanket statement.
Startup can dominate the first answer.
With a large model, loading it into memory and onto the accelerator can cost more than the compute. Cold, OS-warm and restored starts are different costs and are reported separately.
Native deployment
A short dependency list you can inspect.
What the engine sits on — and what it does not.
One binary, driver-only.
The engine process is a single binary that links three shared libraries: the NVIDIA driver (libcuda.so.1), the C library and the math library. No Python runtime, no deep-learning framework, no CUDA toolkit, no cuBLAS or cuDNN. On Apple Silicon the one dependency is Metal.
Weights read in place.
The published checkpoint is read as published, with no converted copy in a private format. Weights are not included in the package; you obtain them from their publisher and they stay under that publisher’s licence.
Sealed, device-bound code.
The GPU code is stored sealed and the key to run it arrives through device-bound activation. The licence is bound to the device and can be revoked. On Linux, root can read a running process’s memory.
Platform support, stated per platform
| Platform | Status on this site | Scope |
|---|---|---|
| Linux x86-64 (native) | Supported | NVIDIA GPU, compute capability 8.0 or newer; a driver that provides libcuda.so.1. No CUDA toolkit and no Python. One GPU runs one model (Qwen package). |
| Windows via WSL2 | Supported | Linux x86-64 under WSL2 with the current NVIDIA Windows driver. |
| Windows (native) | Not a listed target | No native Windows build is listed in the target data. |
| Container | Not documented | No container procedure is published on this site. Scope it in the deployment review rather than assuming it. |
| macOS on Apple Silicon | Supported | M1 or newer, macOS 13 or later, arm64 only. Intel Macs are not supported. |
The NVIDIA package needs the network for activation and licence renewal; there is no documented offline or air-gapped procedure, so it is not offered here as an option. The NVIDIA package also expects ordinary Linux host tools on the PATH (bash, curl, sha256sum, nvidia-smi and others). Requirements and downloads →
Data boundary
Four lines, kept apart.
Each line crosses the boundary differently. They are listed separately so a review can approve or block each one on its own.
1 · Inference
Generation runs on your GPU. For the NVIDIA Qwen package it makes no network calls. Prompts, input media and outputs are not uploaded; the runtime has no media-upload path.
2 · Activation
Needs the network: it registers the device and installs its licence, once per device (veizik activate, browser approval), and the licence is renewed over the network. Activation does not need the model weights. If it does not complete, the device is reported as not activated (exit 6).
3 · Downloads
The veizik command comes from the install script; model packages come from a GitHub release; weights come from their publisher (for Wan, veizik pull wan22 fetches them). Requests to the install script are logged with IP, user-agent, referrer and country, per the privacy policy.
4 · Optional analytics
Website analytics on veizik.com are off until a visitor accepts them. The privacy policy also lists crash and run diagnostics as collected data; read it for the exact runtime scope before approving a deployment. Privacy policy →
Resident, restore, cold
Three starts, three different costs.
Where the model already is decides how long the first response takes. They are never merged into one number.
Not in memory.
The checkpoint is read from disk, weights are prepared in veizik’s own number formats and uploaded to the accelerator. For a 122-billion-parameter model this takes minutes.
In the page cache.
No disk read, but the engine still re-prepares and re-uploads. Tens of seconds for the same model.
Hypercache snapshot.
A prepared, resident snapshot is restored: weights already where the compute needs them, context already built. The cold build remains the fallback. Shipped for supported models only.
Server process.
veizik serve keeps a process alive on 127.0.0.1. The Wan package includes a resident render server that handles request after request in one process.
One record: Qwen3.5-122B-A10B on one RTX 3090
| Start state | Measured |
|---|---|
| Cold, from disk | 230–311 s |
| OS page cache (RAM disk) | 30–46 s |
| Hypercache warm, first token | about 6 s |
One RTX 3090 under WSL2, 24 GB card, Threadripper Pro workstation reading from an SSD array. A development build; the source labels it a record of the load path, not a release speed claim. Cold time is bounded by the drive the model is read from. Separately, a warm restore of gpt-oss-20b on the RTX 3090 is recorded as byte-identical to its cold load. Startup measurements →
Model and hardware scope
What is supported, and what is not measured.
Supported scope as published; no figure is shown where none was measured for an on-premise configuration.
NVIDIA package (Linux x86-64)
Qwen language models and Wan video models. GPU code for sm_80, sm_86, sm_89, sm_90a, sm_100a and sm_120a is included; the Qwen package notes that sm_80, sm_90a and sm_100a code was included but not run in validation. Qwen package disk: about 59 MB; Wan package about 7 MB; the checkpoints you supply are far larger.
Server limits to review
Qwen package: the server is loopback-only (127.0.0.1); to serve other machines put a TLS-terminating reverse proxy in front. Authentication is one shared key per server process (--require-auth), not per-user keys. No usage ledger or per-tenant quotas. Context window 4,096 tokens per request in the validated configuration.
Not measured for this configuration
No multi-user throughput, concurrent-request capacity or fleet-scale figure is published for an on-premise server. Wan speed figures were measured on an RTX 3090 only. Measure these on your own hardware during the evaluation.
Update and verification
Check it before you run it.
Verification uses standard tools against the files you downloaded.
Verify the download.
Each package is published with its file name and SHA-256 on its release page. Check it with sha256sum -c against the published checksums, and use the bundled seal checker.
Verify the dependency claim.
On your own copy, readelf -d or ldd lists the linked libraries, so the list above is a property of the file, not of this page.
Updates.
New versions are published as releases. veizik update is not enabled yet, so version changes are an explicit install you control.
check it on your own download
ldd ./veizik | grep -Ei "python|torch|cublas|cudnn" || echo "none of those are linked"Cost method
Compare on your own inputs.
If you want a cost comparison, use your own inputs; no savings figure is shown here by default.
method, not a result
energy per run (kWh) = measured watts x measured seconds / 3,600,000 cost per run = energy per run x your price per kWh inputs: watts and seconds from your own run; price from your own tariff
Power and time must be measured on your hardware with the same model and settings. Until you enter them there is no number.
Evaluation
Define. Measure. Integrate.
A technical evaluation of your environment, framed as technical demonstrations rather than customer references.
- Define. Target platform (native Linux or WSL2), GPU, model, whether the host may reach the network for activation, and who may reach the server.
- Measure. Cold, OS-warm and restored start on your storage; power and time on your hardware; quality against your reference.
- Integrate. The data-path decision per line, the gaps found (for example per-user authentication, container packaging), and the next decision — yours.
Bring the environment. Define the boundary.
Tell us the platform, the GPU, the model and which of the four data lines your policy allows.
Review an on-premise deployment