How I Built My First Private AI Server Using Second-Hand Hardware
By Sebastián Jaramillo | Activating Creators
You don't need millions of dollars to start building with artificial intelligence. Sometimes, all it takes is second-hand hardware, curiosity, and the determination to discover how far your ideas can go.
Welcome to My Personal AI Lab
Recently, I decided to open the doors to my personal laboratory and share something that usually happens behind the scenes: how I built my own artificial intelligence server using second-hand computer components.
Why did I want to share this experience?
Because I want to challenge a common misconception about artificial intelligence: that building your own AI technology requires enormous investments, cutting-edge equipment, or budgets reserved for large corporations.
The reality is far more interesting.
Today, there are extraordinary opportunities to experiment with artificial intelligence using hardware that once powered major data centers and is now available on the second-hand market.
My personal laboratory is a perfect example.
It didn't begin with a multimillion-dollar infrastructure.
It started with a simple question:
What could I create if I had my own artificial intelligence running on my own servers, entirely under my control?
The answer became a much bigger adventure than I ever imagined.
Giving Technology a Second Life
When companies upgrade their servers, many perfectly functional components enter the second-hand market.
Processors, memory modules, graphics cards, and motherboards that originally cost thousands of dollars can often be purchased for a fraction of their original price.
This opened the door to an exciting possibility: building my own AI server by giving enterprise technology a second life.
My configuration includes two Intel Xeon E5-2620 processors, server-grade RAM, two NVIDIA Tesla V100 GPUs with 32 GB each, and a 750 W power supply.
The Tesla V100 GPUs are particularly interesting because they were originally designed for data centers and demanding artificial intelligence workloads.
Although they belong to an older generation of accelerators, they remain useful for running certain AI models.
With two 32 GB GPUs, I have 64 GB of HBM2 memory distributed across both graphics cards, providing an interesting foundation for experimenting with language models of different sizes.
Of course, having 64 GB distributed across two GPUs doesn't automatically mean they behave like a single 64 GB GPU.
The inference software must properly manage how models and workloads are distributed between them.
And that is precisely where another part of my learning journey began.
How Much Does It Cost to Build a Server Like This?
One of the questions I hear most frequently is:
How much money do you actually need to run your own private artificial intelligence?
To provide a realistic reference, I looked at approximate second-hand hardware prices on eBay for November 2026.
Here is an estimated budget for building a configuration similar to mine:
| Component | Specifications | Estimated Price (USD) |
|---|---|---|
| Processors | 2 Intel Xeon E5-2620 | $20 |
| Motherboard | Supermicro Dual Xeon LGA 2011 | $125 |
| RAM | 256 GB DDR3 ECC | $416 |
| Graphics Cards | 2 NVIDIA Tesla V100 32 GB | $1,400 |
| Power Supply | 750 W | $70 |
| Storage | SSD for the operating system | $40 |
| Cooling and Accessories | Depending on configuration | $100 |
| Estimated Total | AI Server with 64 GB of GPU Memory | $2,171 |
These prices are approximate estimates based on second-hand market references. They do not necessarily reflect what I originally paid and exclude shipping, taxes, import duties, and additional hardware adaptations.
In other words, with approximately $2,200 in components, depending on available deals, you could build an interesting platform for experimenting with private artificial intelligence.
Is that a small amount of money?
That depends on your circumstances.
Is it significantly more affordable than purchasing modern enterprise AI infrastructure?
Absolutely.
And here's the most important part:
You don't need to start with two GPUs or 256 GB of RAM.
You can begin with a much smaller computer and gradually expand your infrastructure as your knowledge and projects grow.
The goal isn't to have the most powerful laboratory from day one.
The goal is to start creating.
One important technical consideration: NVIDIA Tesla V100 GPUs designed for passive cooling require adequate forced airflow. A 750 W power supply must also be evaluated carefully for the complete system's power requirements, connectors, and safety margins. Choosing compatible components is essential.
Building a Server Is Just the Beginning
After assembling the components and getting the server running, I discovered something fundamental.
Hardware is not artificial intelligence.
Hardware is simply the infrastructure that makes running AI possible.
To transform those processors, memory modules, and graphics cards into a system capable of understanding instructions, processing conversations, and executing language models, I needed to understand how AI inference actually works.
That's when I began studying inference engines such as llama.cpp and Strata, along with other technologies.
I explored model quantization, GPU memory management, workload distribution, context processing, and performance optimization.
Every experiment raised new questions.
Why did one model run faster with a particular configuration?
What happened when I distributed its layers across two GPUs?
How could I make better use of the available memory?
How large could a model be before hardware limitations became a problem?
Gradually, I realized that running artificial intelligence involves much more than installing an application and clicking a button.
Behind every response, there is a fascinating world of engineering, mathematics, system architecture, and optimization.
But I also discovered something wonderful:
This knowledge is accessible to anyone willing to learn, experiment, and keep asking questions.
From Studying AI Inference Engines to Developing Xeretron OS
After spending time studying different inference technologies, I began imagining something more ambitious.
What if I could create a platform that brought everything I had learned together and simplified the process of running private artificial intelligence?
I didn't want to depend exclusively on isolated tools.
I wanted an environment where I could manage my AI models, control server resources, integrate additional capabilities, and build intelligent experiences on infrastructure that I owned.
That vision eventually led me to develop Xeretron OS.
Xeretron OS is the platform I developed to integrate and manage artificial intelligence inference within our technology ecosystem.
Its architecture builds on existing inference technologies, including llama.cpp, while incorporating our own components and developments to connect AI capabilities with applications and services.
The goal isn't to reinvent every algorithm from scratch.
Instead, it's about taking advantage of the extraordinary knowledge available through open-source technologies and building new solutions on top of them.
Today, Xeretron is part of a much broader vision at Xerebria Technologies: creating tools that allow people and organizations to use artificial intelligence privately, flexibly, and with greater control over their infrastructure.
And it's exciting to think that much of this journey began by assembling servers and experimenting with used computer components.
Why Does Private Artificial Intelligence Matter?
When we use cloud-based artificial intelligence services, we generally rely on infrastructure managed by third-party providers.
These services offer enormous advantages, but they may also involve recurring costs, vendor dependencies, and questions about how information is processed and stored.
Private AI provides an interesting alternative.
It allows us to execute compatible models on our own hardware and develop applications that don't necessarily need to send every request to an external provider.
For certain use cases, this can provide greater control over data, additional customization options, and more technological independence.
Of course, running your own infrastructure also comes with responsibilities.
Security, maintenance, electricity consumption, cooling, and updates all become part of the equation.
Private artificial intelligence isn't automatically cheaper or better than cloud services in every situation.
But it gives us something valuable:
The freedom to choose how we want to build and use our technology.
And I believe that freedom is extraordinary.
You Don't Need My Laboratory to Get Started
There's something I want to make very clear.
The purpose of sharing this video isn't to convince you to immediately purchase two NVIDIA Tesla V100 GPUs.
You don't even need a dedicated server to begin experimenting with artificial intelligence.
Today, there are compact AI models that can run on personal computers with relatively modest hardware.
You can start by installing an inference engine, running your first local language model, and discovering how everything works.
Then you can experiment with assistants, automation tools, small applications, and your own creative projects.
And if your ideas eventually require more computing power, you can expand your infrastructure.
That's exactly how my journey evolved.
I learned, experimented, made mistakes, fixed problems, and kept moving forward.
Because the real value of my laboratory isn't simply the processors, memory, or graphics cards inside it.
It's everything I learned while building it.
The Next Great Innovation Could Start in Your Bedroom
When we look at the enormous investments made by the world's leading technology companies, it's easy to believe that artificial intelligence belongs exclusively to those with unlimited resources.
But there's another reality.
A curious person with internet access, open-source tools, and a computer can learn technologies that once seemed completely out of reach.
They can develop applications, build intelligent assistants, automate processes, and bring their own ideas to life.
They might not have the resources to train the world's largest AI model from scratch.
But they don't need to.
Creating something extraordinary doesn't always require the most powerful technology.
Sometimes, it simply requires understanding how to make better use of the technology that's already available.
That's one of the most important lessons my personal laboratory has taught me.
And it's one of the main reasons I decided to share this experience.
I don't want you to simply admire what I've built.
I want you to start imagining what you could build.
Because behind every great project is someone who decided to begin, even without having all the answers.
Activating Creators Like You
If you've made it this far, I'd love to invite you to watch the video where I take you inside my personal laboratory and show you how I assembled my artificial intelligence server.
My mission is to share real experiences, knowledge, and practical tools that help people transform their ideas into meaningful projects.
You don't need to be an expert to start.
You need curiosity, a willingness to learn, and the courage to take the first step.
Technology is the tool. You are the creator.
And always remember:
Great ideas don't always come from great laboratories. Sometimes, they begin in a small room, surrounded by cables, second-hand components, and someone brave enough to imagine something different.
Sebastián Jaramillo
Technology Entrepreneur | Founder of Xerebria Technologies
Activating Creators Like You.