A civil litigation attorney built an AI server from eight used 2018-era NVIDIA Tesla V100 data center GPUs. The entire point is privacy. Case files never leave the office, and no client data touches a cloud service. The server runs DeepSeek V4.1 Flash entirely in GPU memory at up to 215 tokens per second for code and 118 tokens per second for legal writing.
Why Offline AI Matters for Law Firms
Attorney-client privilege and data-protection regulations such as GDPR and the U.S. Model Rules of Professional Conduct make it risky for law firms to rely on third-party cloud AI services. Any breach or inadvertent data transfer could expose privileged information, leading to sanctions or loss of client trust. By keeping the model on-premises, the lawyer eliminates the most common vector for data leakage: the internet.
Hardware Choices and Cost Breakdown
The core of the build consists of two four-GPU SXM2 NVLink boards, each priced at roughly $2,400 when purchased. Each board houses four Tesla V100-SXM2-32GB cards, giving a total of eight GPUs. Every V100 card provides 32 GB of HBM2 memory and a bandwidth of 900 GB/s. Inside a board, the four cards communicate over NVLink, creating a combined 128 GB of GPU memory per board. The two boards, however, connect only via the slower PCIe bus.
Beyond the GPUs, the system includes a Gigabyte Threadripper Pro motherboard, a 12-core AMD 3945WX processor, 256 GB of DDR4 RDIMM, and dual 2,800 W power supplies that draw from two ordinary office circuits. The total hardware cost, excluding the labor of troubleshooting, was remarkably low for the performance delivered.
Stability Challenges and How They Were Overcome
Initial testing revealed two fatal PCIe errors when the GPU clocks ran uncapped, causing the host to crash. Each V100 can draw up to 232 W at full power, well above the 180 W rating. One board even showed a visible burn mark on its power connector. To protect the system, the lawyer removed the damaged board and capped the remaining seven cards at 150 W, limiting their clocks to 1,522 MHz.
System RAM also suffered uncorrected ECC errors at the original DDR4-3200 speed. By reducing the memory frequency to DDR4-2666, the errors disappeared, stabilising the platform for continuous operation.
How DeepSeek V4.1 Flash Is Served From GPU Memory
DeepSeek V4.1 Flash runs mostly with 2-bit compression, resulting in weight files that total 248.8 GB. All expert layers of the model reside entirely in GPU memory, while about 110 GB of lookup tables sit in system RAM. Accessing RAM adds roughly 0.12 ms per inference step, a negligible penalty for the speeds achieved.
Layer Distribution Across the Two Boards
The model is split so that the first 20 layers run on one board and the remaining 20 layers run on the second. Each board handles its own heavy traffic over NVLink, meaning only a single handoff per inference step traverses the slower PCIe link. An earlier configuration that spread every layer across all eight cards collapsed performance because the PCIe bandwidth became a bottleneck.
Measured Performance
- Code generation: 215 tokens / second on 1,000-token prompts.
- Legal writing: 118 tokens / second.
- First response for a 128,000-token request takes about one minute.
Practical Use Cases in a Law Office
The server is used for daily first-pass legal work. It can read folders of case records, extract facts, dates, and figures, generate concise summaries, and draft routine documents such as pleadings or discovery requests. All outputs are reviewed by a human attorney before any client communication, ensuring quality and compliance.
Interaction with the AI is handled through a Discord-style interface over SSH, and Retrieval-Augmented Generation (RAG) databases and tool calls are integrated into the workflow. At present the system processes one request at a time, which matches the typical workload of a solo or small-firm practice.
Impact on the Wider Legal Profession
The attorney estimates that 50 % to 60 % of existing AI workflows now skip frontier models entirely, relying instead on smaller, locally-hosted solutions. A second DeepSeek server, built on two NVIDIA DGX Sparks, handles the remaining workload, and the two systems hand tasks to each other as needed.
By demonstrating that a lawyer can assemble a high-performance AI workstation from used hardware and ordinary office power, the build challenges the notion that only large tech companies can run cutting-edge models. For a profession where confidentiality is non-negotiable, the ability to keep data offline is a decisive advantage.
Future Risks and Market Trends
The NVIDIA R580 driver branch is the last to support the Volta architecture that powers the V100. When NVIDIA eventually drops that driver, the hardware will lose official software support, potentially forcing owners to rely on community patches or to upgrade.
Used V100 prices on eBay have already risen from roughly $400 per card to about $900. The narrowing price window means that anyone considering a similar build must act quickly or look for alternative GPU generations such as the newer A100 or H100, which carry higher power requirements and cost.
What Readers Should Take Away
For lawyers and other professionals handling sensitive data, the article shows a viable path to harnessing frontier AI without compromising privacy. The key takeaways are:
- Choose GPU hardware with sufficient memory (32 GB HBM2 per V100) to hold the model weights.
- Use NVLink within a board to avoid PCIe bottlenecks; limit cross-board traffic.
- Cap power draw and clock speeds to stay within safe limits for older cards.
- Drop RAM speed if ECC errors appear; stability outweighs raw bandwidth.
- Plan for driver obsolescence and monitor second-hand GPU market trends.
By following these guidelines, a small office can build a private AI server capable of handling real legal workloads at production speeds.