```html

The Development of Large Language Models (LLMs) and Open Datasets

The development of Large Language Models (LLMs) is at the heart of current advancements in Artificial Intelligence. A crucial factor for the performance of these models is the training data on which they are based. This article highlights the challenges and best practices in creating open datasets for LLM training, based on the research paper "Towards Best Practices for Open Datasets for LLM Training".

The Importance of Open Datasets

Many AI companies train their LLMs with data whose copyrights they do not own. The legality of this practice varies depending on the jurisdiction. While it is permitted under certain conditions in countries like the EU and Japan, the legal situation in the USA, for example, is less clear. Regardless of the legal status, concerns from copyright holders have led to copyright lawsuits. The risk of litigation is often cited as the reason for the increasing secrecy surrounding the composition of training datasets – both by companies and non-profit organizations. This lack of transparency hinders traceability, accountability, and innovation in the AI ecosystem. Researchers, auditors, and affected individuals do not have access to the information they need to understand how AI models work. One solution would be to train LLMs with open-access and public domain data. However, currently (at the time of writing this article), no significantly trained models of this kind exist.

Challenges in Creating Open Datasets

Creating a suitable corpus of open data comes with significant technical and sociological hurdles: * Incomplete and unreliable metadata make it difficult to categorize and select relevant data. * Digitizing physical records is costly and complex. * Diverse legal and technical expertise is required to ensure the relevance and responsibility of the data in a rapidly changing landscape. * The question of consent ("Consent Crisis") in the use of data generated by people poses another challenge.

The Path to Best Practices

The mentioned paper "Towards Best Practices for Open Datasets for LLM Training" emerged from a collaboration between Mozilla and EleutherAI with experts from the open dataset community. It identifies the challenges in creating open datasets and offers practical recommendations for their procurement, processing, management, and publication. These recommendations are based on practical experience and are illustrated with concrete examples. The paper goes beyond the Open Source Initiative (OSI)'s definition of open-source AI by outlining different levels of openness and showing ways for more ethical data management.

Key Issues for the Future of Open AI Training Data

The development towards a future where AI systems can be trained with openly licensed, responsibly curated, and managed data requires: * Collaboration between law, technology, and politics * Investments in metadata standards and digitization * Promotion of a culture of openness and data sharing * Development of mechanisms to ensure consent and data protection * The establishment of clear licensing models and usage rights for training data. Mindverse, as a German provider of an all-in-one platform for AI text, content, images, and research, recognizes the importance of open datasets for the future of AI. By developing customized solutions such as chatbots, voicebots, AI search engines, and knowledge systems, Mindverse contributes to making the potential of AI usable for companies and organizations. The consideration of ethical and legal aspects of data use is a central component of the company philosophy.

Bibliography:

• https://arxiv.org/pdf/2501.08365
• https://huggingface.co/papers/2501.08365
• https://fosdem.org/2025/schedule/event/fosdem-2025-6020-community-insights-best-practices-for-open-datasets-for-llm-training/
• https://arxiv.org/abs/2402.09668
• https://aws.amazon.com/blogs/machine-learning/an-introduction-to-preparing-your-own-dataset-for-llm-training/
• https://wandb.ai/site/articles/training-llms/
• https://www.analyticsvidhya.com/blog/2024/04/open-source-datasets-for-llm-training/
• https://github.com/mlabonne/llm-datasets
• https://kili-technology.com/large-language-models-llms/9-open-sourced-datasets-for-training-large-language-models
• https://ostendorff.org/assets/pdf/ostendorff2024-preprint.pdf

```