Data extraction – Buy vs. Make

OCR – The history of document extraction

For a long time, data extraction from documents was mainly carried out by specialized OCR service providers using rule-based systems. These offered considerable efficiency gains, but came with costs and a certain degree of dependency. Since document processing is often business-critical, many companies are now considering whether developing their own extraction capabilities could offer greater flexibility and control. But is it really worth it, or does it make sense to stick with tried-and-tested solutions?
AI & LLMs – Suddenly everything is easy?
Advances in generative AI (GenAI) make it easier than ever for companies to extract structured data from documents. Artificial intelligence has revealed two new possibilities:

ChatGPT, Gemini, and Co.

Tech giants such as OpenAI and Google offer low-threshold options for extracting documents via API. Companies can choose flexible options that allow them to read out the desired amount of documents. It is also possible to connect to the API, which makes it easy to switch between providers quickly. As AI models are continuously improving, companies expect this solution to offer maximum flexibility.
The advantages can be quickly summarized: flexibility, low costs, and easy connection.

However, this model also has some disadvantages. The first of these is data protection. Documents containing payment data (invoices), personal information, or even medical information can be categorically excluded. Another disadvantage is that this model still involves dependence on a service provider. Even a short service outage (e.g., one hour) can have massive consequences—both in terms of internal dependencies and customer relationships. Even the general flexibility of switching providers is of limited help in this case. In addition, users of Big Tech APIs are exposed to so-called “noisy neighbor” effects. This means that the performance of the system (e.g., speed) depends on the number of requests from all users of the API. Furthermore, Big AI systems are not specialized for specific document types, which is associated with poorer performance in borderline cases and disproportionately high energy consumption.
The disadvantages are therefore data protection concerns, a clear dependency on the service provider, “noisy neighbor” effects, and the fact that these systems are not designed for data extraction from documents.

Building internal capabilities

Companies seeking complete independence from service providers can use open-source AI models to build their own document selection capabilities. By fine-tuning these freely available AI models, companies can tailor them to their specific documents and use cases. This makes them independent of both the security and reliability of a service provider and its product development. Companies thus retain complete control over sensitive data and can easily demonstrate GDPR compliance. For companies that can use such AI capabilities more broadly, this may be the most cost-effective option in the long term. After the initial investment, server costs are comparatively low.

Advantages:

1. Specialization in specific use cases
2. Security and reliability in your own hands
3. Potentially the most cost-effective option in the long term
4. Development of in-house AI capabilities

Disadvantages:

1. High time and analysis costs during training
2. Years until the quality of established providers is achieved (especially in borderline cases)
3. Temporary deterioration of customer experience
4. High initial costs for personnel, training, and infrastructure
5. Continuous training required for quality assurance
6. Limited training data can create competitive disadvantages
7. Oversized server capacities due to peak loads

When does it make sense to use a specialized service provider?

If a company needs a fast, high-quality solution, a specialized service provider is usually the right choice. Service providers such as Gini have trained their AI models for their use cases over many years and are therefore immediately able to handle even borderline cases cleanly. If quality is less important, it depends on risk preference and regulatory requirements in connection with data protection. Some service providers stand out for their strong focus on data protection. Companies should look for certifications such as “Software hosted in Germany,” ISO27001, or BSI C5. In addition, reliability (i.e., uptime) and price are important indicators.