Currently the demo scene supports one vehicle per vehicle type. This is sufficient for testing purposes, but the goal is to have the full demo scene working with multiple vehicles of each type interacting with one another in a distributed fashion.
Supporting multiple vehicles
The primary question that needs to be answered about supporting multiple vehicle instances is whether to go with a service per instance design or a service per vehicle type design. Having services per vehicle instance feels nice architecture wise. Each vehicle has its own hardware service that is running in the simulation which handles the low-level movement code of the vehicle. The system is currently made to support this kind of structure. I just need create multiple instances of the docker containers and pass the right Id on initialization. The only new thing to implement would be a way for the TOS will connect with the right hardware service but that should not be a problem.
Memory
A possible issue with this architecture is with memory. Each gRPC .NET service has footprint of about 30mb.
If each vehicle has its own service the original demo scene with 4 Qc’s, 10 Asc’s and 5 Sc’s would take up ~600mb of ram for just the vehicle services. Beside the hardware services there are the other simulation services and the TOS services and the Unity visualization. In a cloud deployment these figures are not that concerning. Scaling up memory would be done with just one click. However, this project will mostly be deployed locally by developers. They will run the Unity visualization and the simulation concurrently. Having high memory for the simulation could limit the local scalability and therefore the usefulness of this application.
Optimization
Beside this issue there is limited number of optimizations possible with a service per type architecture. If each vehicle has its own service, updating the vehicles would require distinct network calls per vehicle. These would not be able to be batched as each vehicle service is an isolated unit.
Let’s say you have 10 straddle carriers in the simulation. Each has its own service. A simplified implementation may look something like this.
First you request the vehicle data. Then you modify that data. Finally, you send your modified version back to be updated in the data services. There are at least 2 network calls per Sc so 20 calls for all the Straddle carriers. In practice the Sc logic may even require additional requests to be send during the simulation step. You can’t get around the excessive network calls with this architecture. In the long term this will cause performance and scalability issues.
Let’s compare that to a service per vehicle type approach.
In this implementation there is 1 call to get all the vehicle data. After getting all the data, each vehicle is updated (possibly in parallel using multithreading) and then updated with one call. In total 2 calls for all the Sc’s instead of 20 in the previous example. Currently GetAllData is implemented but updating several entities in one call is not implemented yet. But this can be implemented in the future when the need arises when going with the service per vehicle type design.
Because of these two reasons I decided to go with the service per vehicle type design. It requires some adjustment of the services, but it will improve the overall performance, scalability, and overview of the system.
Data-based or call-based
While redesigning the services I thought about going with a full data driven model in the hardware services. Currently the hardware services define protobuffers calls such as DriveTo(targetPosition) and PickUp(containerId). The TOS calls these methods to interact with the simulation. However, this requires a tight coupling between the TOS and simulation systems. A better option could be for the TOS to use the same system as the simulation, where it requests vehicle data and modifies it. For example, DriveTo would change to requesting the ScData and modifying the Target Position field. This would remove the necessity for vehicle services to define a public gRPC service. They would become simple console application that reads and writes to the system without the TOS being directly connected to it.
The call-based design
The data-based design
A problem with this approach would be that events or become more difficult to implement. E.g., PickUp needs to be fired once. So, an event-based system where the data services define some sort of PickUp event needs to be created. Or the TOS would need to set a specific field e.g., ScData.PickUpContainerId and then the hardware service would need to reset it.
A second problem is that implementing this would require further refactoring of the existing systems. I have underestimated the amount of time it takes to refactor the internal logic of the current services. When making any changes to the API or architecture each service internal logic has to be modified. This mostly falls out of my project scope and is exacerbated by the fact that I did not write the original prototype code.
For now, I will not refactor it to data based even though it seems like a better choice, because of the time constraint. I will revisit this later if and when I have time to spare.