Fork of Text Generation Inference.

Published on 2023.08.09 by Nikola Borisov

The text generation inference open source project by huggingface looked like a promising framework for serving large language models (LLM). However, huggingface announced that they will change the license of code with version v1.0.0. While the previous license Apache 2.0 was permissive, the new one is restrictive for our use cases.

Forking the project

We decided to fork the project and continue to maintain it under the Apache 2.0 license. We will continue to contribute to the project and keep it up to date. We will accept pull requests from the community, and we will keep the project truly open source and free to use.

Here is a link to the code: https://github.com/deepinfra/text-generation-inference

We hope that in time a community of other developers and organizations that want to keep this project truly open source will form around it.

License changes mid-flight

Sadly it is becoming more and more common for popular open source projects to change their license after they gain some traction. This happened with MongoDB, Grafana, ElasticSearch, and many others. As a developer, when you decide to adopt a particular open source project, you start investing time and effort into using it. You build your application around it, and you start depending on it. Then, suddenly, the license changes, and you might be forced to find an alternative.

Imagine if meta changes the license of pytorch. Or if tomorrow huggingface decides to change the license of transformers in a similar way to prohibit commercial use.

We believe that the changing of the license of open source projects mid-flight is a unfriendly move towards the community.

If you need any help, just reach out to us on our Discord server.

Related articles

Fine-Tuning vs RAG vs Prompting: 2026 GuideWhen an AI system yields unreliable answers, the root cause could be an unclear system prompt, missing context, poor retrieval quality, or simply using the wrong base model. Teams end up spending weeks experimenting with prompt changes, retrieval-augmented generation (RAG), or fine-tuning to improve response quality. But before deciding which technique to adopt, it is […]

Best Open-Source Multimodal AI Models for Production (2026)Open-source multimodal models have a benchmark problem. Every new release comes with DocVQA and MMMU scores, but those numbers rarely reflect how a model performs on a real workload. A model can lead the leaderboard and still struggle with a scanned invoice, a noisy image, or a multi-step agent workflow. Real-world data is far less […]

Best Kimi K3 SaaS Tools & API PlatformsKimi K3, with its 2.8 trillion parameters and 1M-token context window, represents a significant leap in large language model capabilities. However, deploying, accessing, and managing a model of this scale presents real infrastructure challenges. From optimizing inference latency and managing GPU compute costs to handling multimodal vision capabilities, selecting the right deployment platform is critical […]

View all