Tomorrow’s highly automated environments are expected to support the interoperability and complex cooperation of heterogeneous systems. Dynamic coordination of the work of autonomous and automated platforms with each other and with (semi-) manual processes becomes essential. This will be given special emphasis in environments that also use mobile robots and mobile platforms, such as factories, logistics centers, agriculture and automotive test tracks, and may affect public road, parking facility, mining, or defense applications of the future. We present an architecture for a cloud-based cooperation and coordination framework (Cooperative Autonomy Cloud, CAC), which provides collective perception, decision-making and / or control for heterogeneous platforms as required, at different degrees of system autonomy. The planned CAC will provide the following main services:

• Real-time digital twin (cooperative perception service). Each platform will be able to share their environmental model with each other through logically centralized but physically distributed integration, creating a unified environmental model in real time on the cloud. Additional services running either in the cloud or on the platform can be provided based on this digital twin. The model can cover the full range of static and dynamic elements and place them in context (e.g., HD map lane information, weather, events, moving objects, emergencies).‍
• Resource management, monitoring and teleoperation. Platforms and other resources connected to the system will be available for tasks, missions and operators, and their utilization can be planned, tracked and optimized. For example, the system will allow the operator to take control of a given platform in remote control mode, provided that the platform and the teleoperation peripherals are suitable for the selected purpose, and there is no (higher priority) reservation by another operator or mission.‍
• AI based situational awareness. Relying on the existing perception (even running in the cloud), in addition to the cooperative perception of the environment, the system also performs the cooperative interpretation of the environment and the situation.‍
• AI based decision support. For highly automated mission planning and decision support, we must solve the issues of resource allocation, data sorting and timing, optimization according to objective functions and constraints. All decision points can, of course, be subject to human oversight and approval, and the degree of autonomy can be adjusted dynamically from fully manual mission planning to fully machine mission planning.‍
• Low level control support. The system connects control stations and remote devices, allowing low-level control information to flow.
• ‍Mixed-reality framework. Manages simulation and real-world environments transparently.‍
• System diagnostics.‍
• Data gateway. Data collection, storage, service, analytics support.‍
• Support for standards. Data collection, processing, consumption, mission planning and control subsystems can be connected to the system according to the main industry standards.‍
• Plug-in framework. Algorithms developed / provided by third parties (e.g. perception, control, etc.) will be easily integrated into the system and workflows, including solutions installed in the partner cloud.

The major software services implementing the CAC functionalities listed above are as follows:

• platform-side communication service
• cloud-side communication service
• cloud-side data integration service
• cloud-side resource management service
• cloud-side situation awareness service
• cloud-side decision support service
• cloud data provider service
• data analytics service
• system monitoring and diagnostics service
• environment and system monitoring user interface
• teleoperation user interface