News


NVIDIA Cosmos enables developers to build training-ready datasets for physical AI using just two prompts, leveraging cloud-native blueprints and AI agents.




NVIDIA Cosmos Simplifies Physical AI Dataset Creation with Two Prompts

NVIDIA has introduced a streamlined approach for creating high-quality, diverse training datasets for physical AI through its Cosmos platform. By leveraging just two prompts, developers can curate, augment, and generate datasets using cloud-native blueprints and AI-driven orchestration, significantly reducing time and costs associated with traditional methods.

Streamlining Dataset Creation with NVIDIA Cosmos

Cosmos is an open platform designed to support world model development through frontier models, datasets, frameworks, and skills. A key component of this platform is the NVIDIA Cosmos Data Factory (CDF), which provides five cloud-native blueprints and six composable skills to accelerate data curation, processing, and synthetic data generation (SDG). These blueprints are orchestrated by the CDF Orchestration core skill, enabling seamless workflows without manual intervention.

According to NVIDIA, the platform can deliver up to 60 times faster dataset assembly and reduce costs by 45 times for autonomous vehicle scenario generation, as measured on a cloud server equipped with eight NVIDIA RTX PRO 6000 Blackwell Series GPUs.

Key Use Cases for Physical AI Dataset Development

The Cosmos platform supports five primary use cases for training data creation:

  • Autonomous Vehicle Dataset Generation: This blueprint allows developers to search large databases for driving scenarios, generate novel datasets, and validate them for physical plausibility. For example, a prompt can request transformations of driving scenes into rainy, snowy, or foggy conditions.
  • Image Attribute Augmentation: Developers can create variations of person images by altering attributes like clothing type and color. The system auto-labels the results for use in search and re-identification tasks.
  • Video Data Augmentation: Existing footage can be transformed across different weather, lighting, and environmental conditions while preserving scene structure. The system ensures physical plausibility and auto-labels the augmented videos.
  • Rare Event Video Generation: Using a single source image, this blueprint generates videos of rare events, such as fires or falls, to fill gaps in training datasets.
  • Defect Image Generation: This blueprint creates synthetic images of defects for visual inspection models, addressing the challenge of limited real-world examples.

How It Works: Two Simple Prompts

To begin, developers configure the Cosmos blueprints and skills using a coding agent. The first prompt sets up the environment by cloning the CDF Orchestration repository and running preflight checks. Once the environment is ready, a second prompt is used to execute the desired blueprint for a specific use case, such as generating augmented videos or creating rare event datasets.

For example, a developer working on autonomous vehicle training might use the following prompt: “Search for driving scenes similar to our input video.mp4, transform them into rainy, snowy, and foggy while preserving layout and motion, and check physical plausibility.”

Implications for AI Development

By automating complex data curation and generation tasks, NVIDIA Cosmos significantly lowers the barriers to developing robust physical AI models. This innovation enables faster iteration and broader training coverage, particularly for scenarios that are rare, costly, or dangerous to capture in real life.



Sign in to save this article

Free account: build your research collection and return to read it anytime.

WE WANT YOU!

are you a developer?

  • Proven International Track Record
  • Vertically Integrated Federal Funds
  • Vertically Integrated Tax Credits
  • Vertically Integrated Investors
  • Vertically Integrated Lenders
  • Vertically Integrated Contractors