Writeup on Upgrading the FOSSBot online platform for GSoC

Fri 11 Sep 2026About a month ago~17 min read

In case a reader doesn't know what GSoC stands for, it's Google Summer of Code. In most instances of the text I'll be referring to this using the GSoC abbreviation. Please read more about this great program in the official website.

This post is quite different in nature to the official report I've submitted. I'll concentrate on the main technical challenges I've had during this project, describing what they were, what my initial thinking was, and how I overcame those issues. I'll be quite liberal with sharing screenshots and some recordings, so if you're more of a visual person then you'll appreciate this, I'm sure! I also want to stress out that this is not a tutorial, I am not an expert on any of the content here, and I will be using terminology that might not be familiar to you, without explaining much. I think this now qualifies as a personal blog post.

Good, now with the preliminaries out of the way, let's continue with what the project is even about.

The FOSSBot project

Quotes copied over from https://fossbot.gr/landing-page

FOSSBot is an open source and open design DIY robot that has been developed by Greek Free and Open Source Software (GFOSS) and Harokopio University of Athens (HUA) for educational purposes. The robot does not come alone, but comprises multiple software and system solutions that provide a holistic approach for STEM education at all educational levels.

In this case however, I was working on one of the software components of this project, the FOSSBot Platform:

FOSSBot platform is a web-based platform that provides a set of tools for creating, editing, and running programs for the FOSSBot robot. The platform is designed to be user-friendly and to provide a seamless experience for users of all ages and backgrounds.

A ton of features are present already as I clone the project. A Javascript-based simulator, a Monaco editor, a Blockly editor, as well as an interactive mode that works using a webcam (you can control the simulated robot using hand gestures!).

An API is exposed to these features to handle the robot in the simulator. Here's a very small sample in the Monaco editor: ^d9e0aa

move_forward_distance(1.0)
rotate_90("right")
move_forward_distance(1.0)
rotate_90("left")

Hit and you can see the robot move in the simulator environment! The platform is executing the Python code in real time! At a similar moment during my own testing of the platform – running as simple code as this – I started thinking of the infinite possibilities this platform can support. As well as what a great teaching tool it is.

This is great! Let's now have a small discussion about what I proposed as my GSoC 2026 project.

Proposing 4 new features

I proposed 4 new features, split up into two phases. Phase 1 includes features 1 and 2, phase 2 includes features 3 and 4.

Starting with phase 1, feature 1 is the Simulation physics upgrade, which promised:

• Collision detection
• Rigid body response
• (originally) boundary enforcement
• Good performance on lower-spec computers (similar to what I remember Greek schools used to have, I hope schools now have better computers)

Feature 2, the Stage creator. This ambitious feature was initially promised to deliver:

• Visual stage editor for teachers and students to design, save, and share environments

That's the first half of the coding period. Moving on to phase 2.

Feature 3, the Education Material System. This would include:

• Simple course management system
• Teachers can author sequential educational content
• Students can enroll and progress

Finally, the AI feature of the platform, feature 4, the AI Assistant:

• Helps students debug their code
• Helps teachers draft courses
• Helps teachers create stages
• Is aware of the FOSSBot API

Phew. All these features sound like a natural progression of the website, which is great. But, they're quite ambitious and more complex than they seem, that's for sure. Personally, phase 1 is where I had the most amount of frustration, thinking, and of course, fun! Specifically, feature 1, the simulator upgrade, was the most amount of tuning I had to do to perfect the behaviour of the physics to be as close to what I thought looked great.

Simulator environment

In the proposal I stated that I'd try two libraries and pick the better performing one. First I went with cannon-es, and then rapier. A quick branch was setup named sim-dev – later renamed to sim, which is what I'll be referring to moving on – where I made all simulator-relevant changes.

Checklist of things I need to make sure that work:

1. The floor has a static collider, should not allow the robot to fall
2. Static objects should have colliders
3. The robot should spawn in, stay on top of the floor, move, collide with objects

Simple.

Picking a physics engine

cannon-es was working great until I realized that objects with complex geometries, like an Eiffel tower STL present in some of the original stages, would not generate accurate collision maps. I had to approximate the size of it, using either a 3D bounding box (also known as an axis-aligned bounding box, AABB) or a convex hull. None of these options were okay by me though at the moment. Using the Eiffel tower example, both of these options ment that the robot could not go under the empty space of the tower! It's not just immersion breaking, but it breaks the expectations of the student that's using this platform, and just seems "lower quality".

rapier's convex hull collider felt much better for this.

Through some more testing, I came to realize that I needed to differentiate between two types of objects with colliders: static and dynamic. Static objects needed to have collisions, like the Eiffel tower, but I didn't want them to be moved by the robot. Dynamic ones, like cones, I would like them to move around. This is were I found out about the mesh collider, which did exactly what I needed: collisions for static objects.

At this point I had decided to move on with rapier, so the cannon-es branch is deleted and I've now updated sim. Now, I'm ready to tackle the robot.

Simulating the robot: Part 1

But the screenshots and recodings above already show the robot. Yeah well, this is the initial kinematic version, as used from the original simulator. I hadn't changed much. The robot at this stage is a grouped of objects – its parts – and the entire group has a box collider. This collider moves using kinematic movement. No wheels or engine is being simulated, yet.

I set out for myself to simulate this robot using more advanced methods than the initial method I had.

First I had to import the new and updated FOSSBot v2 design. Importing the parts and aligning them was fun.

Moving forward a bit, the robot now has wheels! Yes it stands perfectly parallel to the ground, no it shouldn't drop the bottom half on a caster ball. OK maybe it should.

Joking aside, this implementation already has multiple issues. First, I definitely do not need this type of collider for the robot. It eats up resources, is complex to handle, and definitely does not help in development. The wireframes look great though. Second, the simulator currently moves only the wheels, which are a set of RAPIER.ColliderDesc.balls with mass and friction. Moving forward increases the linvel of the rigid body. Not only that, but I also start the animation of the mesh, indicating that the wheel is moving. Third, by this point I have very little idea of what is happening and what should be happening.

So I drop out for a bit to re-work the entire simulator architecture that's being used to better align with rapier and completely decouple the code from the legacy simulator.

Simulating the robot: Part 2

New dev environment, new robot. Literally. Meaning I had to re-tune the positions of everything. That's fine. The second screenshot shows something else as well, three box colliders around the robot.

At this point I remembered something from gaming. I remember watching some Dark Souls (not sure which video) analyzing frame data. And then it occurred to me that enemies used capsule colliders. I realized that that's what I need and that it would be great for my use-case. It's easier to tune, better performance-wise, and it provides enough resolution (in my mind this means how accurate the collision to a cone is).

Do you notice something wrong with the video above? Yeah I know the driving looks cool. But it's wrong. This is not how the robot behaves in the real world. It moves by differential steering. Wheels same speed forward → robot moves forward. Not an engine moving the wheels. How does an engine even work in this current configuration. Just remembering this hurts my brain, I was completely lost here.

Bonus error: the stages are much larger than they should be. I hadn't noticed since I was so focused on the robot driving.

Anyways. At the same time I also wanted to fix the ramp issue. I want the robot to be able to go up a ramp, stay in place if stopped, have a harder time on higher degree ramps and be a breeze on lower ones. This means I was optimizing towards two issues at the same time.

Still not done, I am determined that this is the correct way. It just needs some tuning, that's all. Maybe the suspension damping will help.

Here we are. I am finally happy with the driving. I tore down the robot's engine and changed its controller to instead use custom ray cast wheel forces. I'm very happy with the result. Turning is good. Ramp behaviour is what I wanted. Moving feels good. Stopping feels good. Caster works as it should.

Sensors!

Now we have a new challenge, adding sensors. It was much easier compared to the driving. I knew what I had to do. Raycasting. And it worked exactly as I wanted to. Ultrasonic and IRs done.

Now we have an LDR sensor as well. Helper shows if it's activated or not. Works great.

Had fun implementing this, tried various object properties to get this look. Doesn't have headlight functionality yet, here it's just an indicator light, kind of.

The microphone is fun as well. It gives a direction of sound detected, as well as intensity. The light blue globe shows the max range of the microphone. Not sure if it's realistic but it works! Tested sound coming out of the red diamond/gem/ruby thing.

But how does it run?

All of this is on my M3 Pro MacBook. Of course it runs well. It's a modern chip. How does it stand compared to other devices though?

I ran this on as many machines as I could. Many of them don't make sense. I just wanted to show that it runs. Anyway. Here are the results. Each value after the Device column is average moving FPS / first load time in ms.

Fun fact: my Smart TV also ran this. It worked, but I don't think anyone will be using this from a TV. Or maybe you will. I don't know.

Stage builder

Phew. Done with physics and engines and broken robot mechanics. Now on to the next feature.

This was an interesting one because frankly, I don't have any knowledge on creating editors. Especially for 3D stages or objects. I just have a little experience using them. Mostly from Blender, TinkerCAD, and 3D printing software.

So here it is. "Studio" theme, awkward colors and placement of elements. Yep. Well, I'm not a designer so don't judge me too much.

It's a mixup of ideas from the three.js editor , as well as from the PlayCanvas editor (great stuff by the way here, incredibly amazed by the project, and it's open-source!).

All in all it was fun building this and definitely useful, but I think this would greatly benefit from comments from an actual designer. I think I am one step closer to understanding how hard it is to create good UI and UX for software. Makes me appreciate the hard work done by developers from around the world.

Collection of videos

Education material system

Moving on to phase 2. This phase is more straightforward given my experiences with programming.

Can I be honest? Coming into this I thought it would be easy. Coming out of it, I have changed my mind. I've had the delight of working with Open eClass throughout my school years and uni. I've also worked on a project where we needed to setup Moodle. Both great systems. I get why there are so many features in each. It's a hard problem. Making the collaborative (a)synchronous schooling experience accessible, easy for teachers and students, keeping appropriate backups, showing clearly what happened after each click, making sure it works on a variety of network connections. I could go on, but I'm sure you get the point.

So yeah, in similar fashion to the conclusion I gave in [[#Stage builder]], I have to say that this will immensely benefit from user feedback. I know there's much to fix.

A sample or two courses should be available I think for students and teachers. A manual of sorts. I'm sure I'll have the pleasure to do that at some point.

FOSSBot Buddy

Hey buddy!

This is the final feature of my project. An AI assistant that helps teachers and users.

It's an interesting tool to have. Not so much in my opinion for students. Therefore, all usage is closed by default and only teachers in communication with the admins will be able to allow access to the feature.

I've tested this with OpenAI (API), OpenRouter, a local llama.cpp instance, and WebLLM. All worked quite well in this instance, especially for coding and course creation.

Stage creation struggles quite a bit and it's understandable. The entire system would need to better support the AI integration into the 3D simulator. In my opinion, this is something that needs careful changes and a ton of testing. Even the frontier models at the time of writing this (GPT-6 and Fable 5.1, early September 2026) are not great at 3D design. And if you want to test them anyway, well, you will need deep pockets, which is something I cannot afford to do.

I think for just getting you started it's great, drafting up ideas and the user taking their time improving, day by day.

AI tools

It's time to address the elephant in the room. Coding with AI.

After a break from programming for some time, I came back to work on this project. I was quite behind on things. Keep in mind, what I used to do before this was to ask in VSCode's CoPilot tab questions about my code, help me understand, generate code, etc. But the entire process was guided, meaning that I would oversee everything that went into the work. I didn't leave any agent writing code. I didn't think the technology was mature enough for that.

So coming back I decided to see what the fuss is and take this opportunity to add agents in my toolset.

I tried many tools, such as OpenCode, gemini-cli, Codex, Claude Code, and pi.

I was very skeptical initially. I didn't like the feel of letting an LLM around my system, deciding what to keep, how to change something. I still don't know if I like it entirely. But as some point I just let go. Idea after idea, sessions in parallel. Open laptop well after my normal sleeping schedules. Minding my token usage, working around resets. I got too excited, and hooked. If you've read pieces from other developers, or if you've gone through this you know the feeling and what I'm mostly about to say.

Personal feelings aside, I decided to use agents to organize my work in this project, as well as generate code. Everything the tools produced I manually checked and verified to understand what was done. I was something akin to QA and reviewer. Many times, especially during the [[#Simulator environment]] feature, I had to completely stop prompting and think hard on what needed to be done exactly. Research exactly what I had to do, call my mentor, write on paper. I am glad to say that I was and am still able to do this, and I can understand when a result is bad, good, or mediocre. Not for everything, many things will fly under my radar.

For the level of projects I am currently doing I'd say this mixture of agentic programming with manual overview is acceptable. If any part of the projects I work on are handling important information, I revert to what I did before this: advisory role to coding agents, manual tooling, debugging (I'm big on print) until the feature is done. I honestly don't know if this is the right way of handling things.

Conclusion

I had a great time working on this project. If anyone reading this is a student and hasn't participated yet, please do so. Check out projects, use them yourself, think of features, propose them. I am sure you'll get accepted and you'll have the privilege to work on a GSoC project yourself.

I'm happy to say that I'm still contributing to the project, and will continue to do so!

If you have any questions or comments please don't hesitate to contact me.