This internship assignment revolves around creating a simulation of a container terminal operating system. The simulation uses a microservice architecture and can be connected to an existing 3D visualizer. For more details on the assignment read the assignment page
Creating a Microservice Prototype
For the first week of my internship I decided to analyse the existing code base and its internal communication through prototyping. Usually interns start of with writing their project plan. I reasoned that writing the project plan would go smoother if I had some insight in the existing codebase and how the communication between the different layers occurs. So I made a plan to analyse the existing codebase by prototyping a small microservice application.
The goal of this project is to analyse the current functionality, analyse the intercommunication between classes and to make a proof of concept of how this communication could work in microservice architecture.
The steps for this project went went like this:
- Separate the existing C# codebase in distinct projects.
- Define the dependencies between the different projects.
- Separate the single application into several applications.
- Refactor the internal communication to gRPC.
- Build and run the services in a docker environment. (because of time constraints I did not manage to do this part. more on this later)
Separating the existing codebase
Monday I started with this project. I setup the visualization and simulation projects locally. I cloned the visualization and simulation code. I could both build and run the applications and have them connect to each other. I started analyzing the simulation codebase. The codebase is a single project C# solution. This means that any class can reference any other class in the whole codebase. This would be a difficult starting point for the microservice project. As it would be difficult to find the distinct layers. Therefore I decided to first separate the project in distinct C# projects. Fortunately the internal structure of the codebase is well organized in folders with minimal outside references. I used these folder structures as guidelines when defining the new projects. There were some circular dependencies. These were fixed by moving some code and method calls.
By the second day I had separated the project in 10 C# projects with clear dependencies. 3 of the projects were shared libraries. They provided the data models and utilities functions. The others project had clear dependencies and were looking more like service. Each project had one controller class that represented the API. The codebase wasn’t fully separated. I left the workers together in one project as they all share base classes. This will have to be fixed for the microservice architecture as I want to have each vehicle to be an individual service as this is one of the goals of the final product.
I made a dependency diagram to visualize the dependencies after separating the codebase.
Each of these blocks are C# projects in the separated version of the simulator. They will have to be transformed
into
individual services.
Services with gRPC
Wednesday morning I started with creating the individual services. I decided to use gRPC as I already have some experience with it, has good C# support and would require minimal amount of refactoring compared to a messagebus approach. I defined the API in the protobuffers messages and services. I wrapped the existing classes in services, wrote some translation code to translate between the generated protobuffers models and the existing data layer models. There were some minor issues but most of this work was relatively straight forward. I tested the services using BloomRPC.
Bug fixing
Thursday I managed to finish all the services. The simulation did not yet work, there were some issues. Friday morning I went looking for these issues in the hope of having something working for the demo in the afternoon. I found some issues most of which were relatively minor such as using the wrong API call or forgetting to translate null objects properly.
There was one persistent issue that was harder to fix. Straddle carriers (a type of vehicle) would try to pick up containers from the wrong location. They would move to the water instead of to the crane location. This issue turned out to be because of my misunderstanding of a specific class named PositionProvider. This class holds the position information about the highways and stack area. I assumed this class was static as it only had initialize methods. So I put it in the shared library section. I had not foreseen that the cranes were storing their location in here as well. Because this information would not be initialized in the other services the vehicles would move to x=0,y=0 to pick up the containers which was near the water.
In the original simulation the cranes would initialize their position in the PositionProvider and update this whenever they would move in the X position. This would not be possible now as each service has a local instance of this class running. For now I fixed by let it remain static and manually initializing the crane position in the PositionProvider to their start position. In the demo the crane would only move in the X axis after 10 minutes. So it would not be a big issue for this prototype.
Deployment
I did not have the time to make a proper docker deployment so I decided to make a simple bat file that would execute all the services with an interval between certain services to have the proper dependencies loaded for each service. To keep things simple I decided to run one vehicle service per vehicle type. Each service that has an public API has a hardcoded port number assigned. Something like Kubernetes service discovery would be helpful here but this seems like to much for this prototype and I wanted to have it finished by Friday.
Demo
Friday afternoon I demo'd the project to other interns and employees. It went quite well and they were impressed with what I was able to achieve the first week. Here is a short video of what I demo'd.
Next week I will try to finish a first version of the project plan.