Ghost Autonomy Engine

A flexible AI software platform enabling OEMs to bring L4 autonomy to tomorrow’s consumer vehicles.

Autonomy Software

AI software is at the heart of how Ghost sees, understands, and drives.  Ghost Autonomy Software is responsible for obstacle and scene perception, route planning and driving, and understanding driver intent.
Perception
Multiple neural networks process raw camera and radar inputs to develop an accurate understanding of relevant vehicles, obstacles, and the road configuration in real-time.
KineticFlow™ vision neural network

A universal video network that combines stereo disparity and mono motion to calculate the distance, absolute velocity, and trajectory of objects both near and far without requiring object recognition.

Explore KineticFlow
Radar neural network

A novel approach to processing raw HD radar outputs extends perception distance and provides resiliency in inclement weather, poor lighting, and occlusion scenarios as an independent sensor modality.

Scene and lane canonicalization neural network

Interprets the real-time environment and identifies drivable paths by detecting road markers and lane semantics and combining them with vision and radar data.

Driving

The driving program analyzes the outputs from Perception and performs the task of driving, executing standard and defensive maneuvers to optimize safety, comfort, and route.

Driving maneuvers

From lane centering and distance keeping to merging and changing lanes, Ghost executes driving maneuvers by precisely actuating the steering, accelerator, and brake.

Defensive maneuvers

Aided by 360° perception and reaction times that are three times faster than a human driver, Ghost is capable of aggressive braking and swerving to avoid dangerous obstacles and events on the road.

Navigation and route optimization

Ghost will pass slow vehicles, drive in eligible HOV lanes, and use OTA connectivity and real-time traffic data to continuously optimize routing decisions.

Safety

Multiple layers of software and hardware redundancy enable Ghost to drive safely without relying on a human backup, even in the event of failures or occlusions.

Fail-operational resiliency

With high availability software and redundant sensors, sensor modalities, and processors, Ghost can continue driving safely in the case of sensor occlusions, dropped frames, or even full sensor or compute failure.

Safe stop

In case of the most severe failures, Ghost will achieve a minimal risk condition with an independent driving computer designed to bring the car to a safe stop without human intervention even if the primary system is compromised.

Operational design domain

The approved operational design domain (ODD) is actively enforced and can be modified via OTA updates to expand driving areas and use cases and execute targeted restrictions if ever necessary.

Driver Intelligence

Using an in-cabin camera equipped with artificial intelligence, Ghost is developing sophisticated driver intent perception models capable of transforming the interaction between car and driver.

Driver monitoring neural network

Ghost monitors steering wheel and pedal inputs as well as the driver's hand positions, head and skeletal posture, and eye gaze to understand when to transition driving responsibilities to and from the human driver.

Driver intent neural network

As a L4 system, Ghost does not require driver attention or intervention when engaged. Equipped with driver intent perception that can disambiguate accidental input on the car controls from deliberate driving activity, Ghost seamlessly begins driving when the driver simply lets go of the wheel and pedals. Likewise, the driver can take back control at any time by taking the wheel or pressing the pedals.

Ghost OS

A real-time operating environment responsible for high-performance execution of the Autonomy Software.

Formally verified critical runtime components

Verified to be bug-free using formal software verification methods adapted from aerospace and defense.

Sensor and controller abstraction

Works seamlessly with a range of sensor and compute configurations, enabling OEMs to customize hardware across models and trim levels and deliver future hardware upgrades.

Security-hardened OS

Implements best practices for hardening, securing communications, and intrusion detection to manage cybersecurity risk.

Software update, versioning, and entitlement

Enables secure update and rollback of software versions and facilitates unique feature entitlement per subscriber.

Event and exception capture and upload

Sends unusual road scenes and system exceptions to Autonomy Cloud for training and analysis.

Intra-vehicle communications

Securely directs in-car data flow between sensors, driving computer, and vehicle controls.

Reference Hardware

The Autonomy Engine is optimized to run on a scalable Reference Hardware stack, which serves as both a co-development platform for test vehicles and a blueprint for vehicle-specific sensors and ECUs designed in partnership with OEMs and Tier 1 suppliers.
Back Cameras
Driving Computer
OTA Communications Hub
Side Cameras
Side Cameras
Front Stereo HD Cameras
HMI
Vehicle Controller
Front HD Radar

Design Principles

Ghost Reference Hardware represents a minimum specification of requirements for Ghost-enabled vehicles to ensure safety and performance.  Individual implementations may be extended with additional sensors and capabilities during OEM-specific design programs.
High availability
Multiple layers of sensor and compute redundancy enable safe, fail-operational design.
Mass-market economics
High-volume sensors and general-purpose compute enable scalable economics.
Low power
Optimized for <100W power budget, ideal for preserving electric battery range and fuel economy.
Modularity and upgradability
Designed to support vehicle-specific packaging and ongoing evergreen sensor and compute upgrades.
Automotive grade
Designed to meet automotive shock, vibration, temperature, and electrical requirements.

Autonomy Studio

A combination of APIs, SDKs, and professional services to help OEMs integrate, customize, and extend the Ghost Autonomy Engine to their models and brand.
APIs

APIs in both the car and the cloud facilitate deep integration across core systems ranging from the CAN bus to OTA updates to the HMI, delivering data in real-time and powering immersive visual experiences.

Driving Studio

Sports coupes and heavy-duty pickup trucks are meant to drive differently. Develop custom vehicle and brand-specific integrations, maneuvers, and driving styles without having to build a new software version for each implementation.

AI Studio

Leverage the complete sensor package and Ghost’s AI team to build experiences beyond autonomy, from facial recognition-enabled personalization to gesture-based cabin controls.

Design Studio

Make each vehicle brand unique by partnering with Ghost’s in-house design function to explore, test, and implement deeply embedded HMI, haptic, auditory, mobile, and voice control features.

Autonomy Cloud

Connecting every Ghost-enabled car with one of the world’s most advanced AI supercomputers to deliver continuous safety and performance improvements on the road.
Ghost Connect

The communications hub delivering connected and mobile experiences to the Ghost-enabled fleet and a platform for collaboration between Ghost and OEM partners.

OTA updates

Frequent over-the-air software updates improve the driving experience, deliver new capabilities, and expand the ODD.

Configuration and entitlement

Manages feature entitlement per vehicle, enabling and/or gating capabilities as users modify their subscriptions.

Data collection and analytics

Test and production vehicles capture and upload relevant sensor and driving data in privacy-compliant fashion for training and analysis, enabling constant iteration based upon unusual road scenes, hardware and software exceptions, car and driver behavior, and more.

Driver account and profile management

Syncs with OEM infrastructure for account and subscription management and driver personalization.

Ghost Train

Centralized AI supercomputer training infrastructure for joint neural network development and validation.

Large-scale AI training and validation

Neural networks are rapidly trained and verified using auto-labeled training data and can be rapidly re-trained to ensure compatibility with OEM-specific vehicles and sensors.

350,000+ GPU cores

Custom-built GPU nodes for AI training plus dedicated high-bandwidth public cloud connections for leveraging infinitely scalable cloud compute.

Training automation framework

Ghost’s training and automation practices enable a large team of data scientists to work on many models in parallel.

Risk & Compliance

Ghost believes in working closely with the regulatory community and pursues system validation jointly with OEM partners.

Joint validation infrastructure

Ghost tools and resources support OEM processes for validation, certification, regulatory compliance, and reporting.

Audit and investigation

To support OEM quality and risk mitigation efforts, Ghost preserves neural network versioning for proactive risk assessment and real-time issue management.

ODD controls

The operational design domain can be modified via OTA updates to expand to approved driving areas and use cases and execute targeted restrictions if ever necessary.

Get the Autonomy Engine Tech Brief
Thank you
If the download did not start, click here to view the pdf.
Oops! Something went wrong while submitting the form.
Perception
Driving
Safety
Driver Intelligence
Ghost OS
Ghost Connect
Ghost Train
Risk & Compliance