Eunha

XR Interaction Toolkit (Unity VR Pathway)

Sat, 20 Dec 2025 00:00:00 GMT

https://learn.unity.com/pathway/vr-development

Materials

XR HMD
- If you use a meta quest, please install Meta Horizon Link in advance to connect your unity with HMD.
Unity (recommend LTS version)

Basic Setting

Download Packages
- From the main menu, select Window > Package Management > Package Manager
XR Plugin Management XR Interaction Toolkit OpenXR Plugin Universal Render Pipeline
Make a XR Origin
- From your hierarchy, click right and select XR > XR Origin
Run the app
1. With the Device Simulator

C# Studying

Tue, 16 Dec 2025 00:00:00 GMT

C# Studying

This post is for my C# studying.

1. Variables and Data Types

What is a variable?

A variable is a named space in memory that stores a value. In C#, every variable must have a type, which determines:

how much memory it uses
what kind of values it can store

Value Types

Common value types include:

Example:

int level = 100;
float speed = 3.14f;
bool isActive = true;
char grade = 'A';

Key point: The type decides what operations are allowed. You cannot treat an int like a bool, even if the value “looks similar”.

Variable Initialization

Variables can be:

initialized immediately
declared first, assigned later

Example:

int a1, a2, a3;
a1 = 10;

int b = 20;

If you forget to initialize, the compiler will usually stop you. This is intentional: C# avoids “garbage values”.

2. Operators

Arithmetic Operators

From the operator lecture :

+ addition
- subtraction
* multiplication
/ division
% remainder

Example:

int num1 = 5;
int num2 = 2;

int sum = num1 + num2;   // 7
int mod = num1 % num2;   // 1

Division and Casting

If both operands are integers, integer division happens.

int a = 5;
int b = 2;
float result = a / b;   // result = 2 (not 2.5)

Correct way:

float result = (float)a / b; // 2.5

3. Conditional Statements

if / else

Conditionals allow the program to choose a path.

int num = 11;

if (num > 10)
{
    Debug.Log("num is greater than 10");
}
else
{
    Debug.Log("num is 10 or less");
}

Conditions must evaluate to bool.

Logical Operators

&& AND
|| OR

Example:

if (num > 0 && num < 20)
{
    Debug.Log("num is between 1 and 19");
}

Important detail: C# uses short-circuit evaluation. The second condition may not be evaluated if the first already determines the result.

switch and enum

Used when states are discrete and named.

enum STATE
{
    NONE = 0,
    INIT, //1
    PLAY = 100, //100
    OVER, //101

}

private int num = 0;
    private STATE currentState = STATE.INIT;

switch(num)
{
    case 100:
        {
            Debug.Log("switch 100");
        }
        break;
    case 90:
        {
            Debug.Log("switch 90");
        }
        break;
    default:
        Debug.Log("switch default");
        break;
}

This appears in the later control-flow section.

4. Loops

for loop

Used when the number of iterations is known.

for (int i = 0; i < 5; i++)
{
    Debug.Log(i); // 0 1 2 3 4
}

while / do-while

Used when the stopping condition is more important than the count.

Key difference:

while checks first
do-while runs at least once

int num1 = 0;
while (num1 < 10)
{
    Console.WriteLine(num1);
    num1++;
}

int num2 = 0;
do
{
    Console.WriteLine(num2);
    num2++;
}
while (num2 < 5);

break and continue

for (int i = 0; i < 10; i++)
{
    if (i % 2 == 0) continue;
    if (i > 8) break;
    Debug.Log(i);
}

5. Arrays

Why arrays exist

Arrays store multiple values of the same type in one structure.

int[] scores = { 90, 70, 50 };

Access by index:

scores[0]; // 90

Looping is almost always required:

for (int i = 0; i < scores.Length; i++)
{
    Debug.Log(scores[i]);
}

2D Arrays

Used when data has rows and columns.

int[,] grid = new int[2, 3];

Access:

grid[0, 1];

6. Methods (Functions)

Why methods matter

Methods:

reduce duplication
make code readable
isolate logic

Important rule: A method should do one thing. If it does more, it probably should be split.

Example

void Print()
{
    Debug.Log("Hello World");
}

int MaxInt()
{
    return int.MaxValue;
}

long Sum(int a, int b)
{
    return a + b;
}

MCP with Blender & Unity

Mon, 11 Aug 2025 00:00:00 GMT

MCP in Blender & Unity with Claude AI

Recently, I experimented with MCP (Model Context Protocol) to connect 3D tools like Blender and Unity with Claude AI. My goal was straightforward: see if AI could directly manipulate or inspect my 3D environments in real time. This post is less of a polished how-to and more of a developer log — what worked, what broke, and what I learned.

Understanding MCP

MCP (Model Context Protocol) is an open protocol that standardizes how applications talk to large language models (LLMs).

Key ideas I had to wrap my head around:

Two-way communication between AI and programs.
Ability to create, modify, or delete 3D objects.
Editing materials, inspecting entire scenes, and even executing code through the AI.

💡 Why this matters: Instead of AI being a “detached helper,” MCP turns it into an active collaborator inside your workflow. That was the hook for me.

Tools & Sources

Blender 3.0+
Unity 6+
Python 3.10+ (Blender MCP), 3.12+ (Unity MCP)
Blender MCP GitHub
Unity MCP GitHub
Claude AI App
UV Package Manager

Step-by-Step Setup

Setup

1. UV Package Manager

This is required for both Blender and Unity MCP.

Open terminal and install UV Package Manager.
If the first command failed, switch to the alternative one provided in the docs.

💡 Why: MCP depends on the package manager for dependency handling. Without this, nothing connects properly.

Blender MCP Setup

1. Addon Installation

Download addon.py from Blender-MCP GitHub.
In Blender, go to Edit → Preferences → Add-ons.
Click Install from Disk and select the addon.py file.
Enable the addon by checking the box next to "Interface: Blender MCP".

💡 Why: This gives Blender a sidebar tab for MCP. Without the addon, Blender doesn’t know how to “talk” to Claude.

2. Blender MCP Tab

Open the 3D View sidebar (press N).
Find the BlenderMCP tab.
Check Poly Haven if you want asset streaming.
Click Connect to Claude.

💡 Why: This is where you actually link Blender’s context to Claude. It felt surreal seeing AI recognize what was in my scene.

3. Claude AI Config

Open Claude AI App and go Settings → Developer → Edit Config.
Copy and paste the below code in claude_desktop_config file.

{
  "mcpServers": {
    "blender": {
      "command": "uvx",
      "args": ["blender-mcp"]
    }
  }
}

Restart the computer.

💡 Why: This step gives Claude the bridge it needs to process MCP requests. Took me a few tries before it clicked.

Video

Unity MCP Setup

1. Python Path Setup (Windows)

Open Python file and click “Open File Location”.
Copy the file path.
Open “Edit the system environment variables”.
Click Environment Variables.
Click Path in System variables and Edit.
Paste the file path.

2. Package Installation

In Unity, open Window → Package Manager.
Click + → Add package from Git URL....
Paste the link: https://github.com/CoplayDev/unity-mcp.git?path=/UnityMcpBridge.
Click Add.

💡 Why: MCP for Unity isn’t built-in — GitHub package installation is the only way to get it.

3. MCP Auto-Setup

Go Window → Unity MCP.
Click Auto-Setup.
Look for a green status indicator🟢 and "Connected ✓". (This attempts to modify the MCP Client's config file automatically).
Restart the computer.

💡 Why: Auto-setup edits the MCP client config automatically. Saves time and avoids manually editing JSON files.

Video

Discussion & Reflections

Challenges: MCP requires detailed context to be useful. If you don’t feed enough scene info, the AI’s edits are random or incomplete.
What worked: Once connected, I could spawn and modify objects in Blender directly through Claude — faster than manual modeling tweaks.
Limitations: MCP helps speed up repetitive tasks. But currently it doesn’t replace a human designer at all. You still need to think critically about scene composition.

Future Outlook

Game Development: AI-assisted real-time prototyping inside Unity.
Simulation: Rapid environment adjustments for training data.
VR/AR Content Creation: AI could manage assets or materials while you focus on immersion.
Labor Impact: Might reduce grunt work, but creative direction still belongs to humans.

Blender Tutorial

Fri, 08 Aug 2025 00:00:00 GMT

My Blender Tutorial Experience

I recently dove into 3D modeling with Blender, a free yet insanely powerful tool. My goal? Prepare for future Unity projects and finally get comfortable creating things from scratch in 3D. To kick things off, I followed two YouTube tutorials:

Understanding the Basics

Before starting, I clarified essential 3D modeling concepts.

Term	Description
`Mesh`	A combination of vertices (points), edges (lines), faces (polygons), or any combination of these.
`Geometry`	Any 3D object with position data, including meshes, curves, instances, volumes, etc.
`Instance`	A duplicate of an object sharing the same underlying data as the original.
`Shader`	A program controlling how light interacts with a surface, affecting color, reflection, transparency, and emission. In Blender, it’s often used interchangeably with `material`.

💡 Why it matters: Blender can feel like a maze at first. Understanding meshes, modifiers, and basic geometry gave me a clear roadmap—I suddenly knew what to click, move, and tweak.

Tools & Resources

Blender: Download v4.4
YouTube Tutorials:
- Beginner Tutorial
- Floor Plan Tutorial

Essential Shortcuts

Shortcut	Action
`Middle Mouse`	Orbit around scene
`Shift + Middle Mouse`	Pan view
`Mouse Wheel Scroll`	Zoom in/out
`Tab`	Switch between Object/Edit Mode
`N`	Toggle Sidebar

For Object Mode:

Shortcut	Action
`T`	Toggle Toolbar
`G`	Move objects
`R`	Rotate objects
`S`	Scale (resize) objects
`Shift + A`	Add object
`Alt` (when adding)	Add object in perfect form (e.g., perfect cube or triangle)
`Ctrl + L`	Link properties between objects

💡 Tip: For more shortcuts, check Blender’s Keymap Documentation.

Recommended Setup

Go to Edit > Preferences > Interface and adjust the Resolution Scale
💡 Why: Makes Blender easier to work with depending on your screen size.
Check your GPU under System > Cycles Render Devices > CUDA
💡 Why: Ensures hardware acceleration for faster rendering.
Increase Undo Steps to 100 in System > Memory & Limits
💡 Why: Allows more flexibility when correcting mistakes.

Step-by-Step Experience

Tutorial 1: Blender Beginner Tutorial

This tutorial covered the fundamentals:

Navigating Blender’s viewport
Differences between Edit Mode and Object Mode
Building a simple object step by step

1. Cookie Base

Add a Cylinder (Shift + A > Mesh > Cylinder).
Scale (S) to flatten into a cookie shape.
Right-click → Shade Smooth for a softer look.
Rename it Cookie in the Outliner to stay organized.

2. Chocolate Chips

Add a UV Sphere (Shift + A > Mesh > UV Sphere)
Scale down (S) to chip size
Position using Move (G)
Duplicate chips (Shift + D) and place around the cookie

3. Tray

Add a Cube (Shift + A > Mesh > Cube)
Scale and move to sit under cookie 1insetFace
Enter Edit Mode (Tab), select top face, use Inset Faces (I)
Extrude inner face down (E) to create tray depth
Move the tray under the cookie.

4. Materials & Colors

Add material in Material Properties → New
Choose Base Color for cookie, chips, and tray
To quickly assign the same material, select chips → last chip with material → Ctrl + L > Materials

5. Lighting & Rendering

Click Viewport Shading: Rendered
Delete default light → Add Area Light (Shift + A > Light > Area)
Position camera (0 on numpad, enable Camera to View)
Set render engine to Cycles for realism
Render with F12 → Save via Image > Save As

Tutorial 2: Blender Floor Plan Tutorial

This tutorial focused on combining objects and creating a full scene:

Importing and scaling floor plan images
Modeling floors and walls
Applying materials, lighting, and render settings

1. Setup

Download a floor plan image
Open Blender, select all (A) and delete (X) default objects
Import image → Creates an Empty object
Reset location & rotation: Alt + G, Alt + R
Switch to Top Orthographic View (7 on numpad or click z in xyz)
Set Units to Metric (Scene Properties → Units → Metric → Meters)

2. Floor & Walls

Add Plane (Shift + A > Mesh > Plane)
Rename to Walls and set dimensions in Item Sidebar → Apply scale (Ctrl + A > Scale)
Click walls and set origin (Shift + S > Cursor to Selected & Right Click > Set Origin > Origin to 3D Cursor) and clear any translation by Alt + G
In Edit Mode, delete face (X > Only Faces)
In Object Mode, move the floor plan image fit right in the Wall line
Delete three vertices in Walls (X > Vertices) ![extrudevertices]
In Edit Mode, extrude vertices (E) along X/Y to match outer walls
Extrude inner walls for rooms too
Duplicate Walls and rename the duplicated one as Floor and hide the Walls by clicking eye icon
For the same floor, delete some overlap unnecessary vertices
For each room, select all of the vertices and make a face(F).

3. Finalizing & Rendering

Duplicate floor (Shift + D > Right Click) and rename Background
Hide Floor and Walls
In Background, delete all inner vertices
Extrude everything and scale (A > E > S)
Hide Floor and Background. Select Walls → Extrude upwards along Z-axis (E > Z)
See walls' orientation (Overlays menu > Face Orientation) Modifier.
Add Solidify Modifier (Modifier > Add Modifier > Solidify) and change some settings (Mode Complex, Thickness)
Click Monitor Icon in Modifier
Fix flipped faces (Shift + N) (Blue = outter, Red = inner)
Click monitor icon again
Orientation overlay off (Overlays menu > Face Orientation)
Save your work (Ctrl + S)

Summary

At the end of the day, it wasn’t just about a cookie or a floor plan. These tutorials taught me how to bring any idea to life in 3D. Modeling, texturing, lighting—once you get the hang of these, you’re ready for bigger projects, whether it’s games, architectural visualization, or AR. Honestly, seeing something I imagined appear on screen made me even more excited to dive into Unity. In 3D, the only limit really is your imagination.

💡 Final Tip: Always organize objects, name them clearly, and save iterations frequently. This keeps projects manageable as scenes get complex.

MR Meta Quest Step-by-Step Tutorial

Wed, 06 Aug 2025 00:00:00 GMT

Mixed Reality on Meta Quest 3

I've been diving into Mixed Reality (MR) recently, and I wanted to experiment with a simple Passthrough prototype on my Meta Quest 3. The goal was simple but oddly satisfying: see my real room through the headset, spawn virtual balls, and watch them bounce off walls and floors. Honestly, it felt a bit like playing catch with my own furniture. This post is my personal developer log — not a polished tutorial. I’ll walk through what I did, why I did it, what broke (and it did), and what I learned along the way.

Understanding the Basics

Before jumping into Unity, I clarified the essential concepts.

Mixed Reality (MR): blends virtual content with the real world.
Extended Reality (XR): umbrella term for VR, AR, and MR.
Passthrough: lets you see the real world through the Quest's cameras, layered with virtual objects.

💡Why this matters: MR is not just "turn on a camera." Understanding this early saves time when configuring cameras, layers, and interactions in Unity.

Tools

Unity: v.2020.3+
Meta Quest 3
USB-c Cable (for Building)
Oculus Integration Package from Unity Asset Store

I used 2020.3.1f Unity version. Passthrough and OVRManager behaves slightly differently depending on Unity version. Specifying this ensures anyone trying to replicate my steps hits the same quirks and solutions I did.

Step-by-Step Tutorials

Setup

1. Enable Developer Mode

Create a Meta Developer Account in Meta.
Turn on Developer Mode in the Meta Horizon app

2. Setting Up Unity for Oculus Development

Go to Edit → Project Settings → XR Plug-in Management, enable Oculus for both Windows and Android.
Install Oculus Integration from the Unity Asset Store.
Run Tools → Project Setup Tool → Fix All & Apply All (this actually fixed more problems than I expected).
Switch platform in File → Build Settings → Android.

💡 Why: Quest 3 runs Android. Without the XR plugin + Oculus package, Unity can’t talk to the headset — I learned this the hard way when nothing showed up in my first test build.

Connecting the Headset & Building

Method 1: AirLink (Wireless)

convenient, but slightly blurry visuals and longer builds — kind of frustrating when you’re in the zone

Turn on Developer settings in Quest (Settings → Beta)
Connect Quest to PC via AirLink
Run from Unity. (May cause longer load times)

Method 2 (Recommended): USB-C (Wired)

more stable, faster builds

Connect Quest via USB-C cable
In Unity, go to files → build settings and click Build and Run.

Passthrough

1. Camera Configuration

Delete the default Main Camera.
Add OVRCameraRig prefab.
In OVRManager (inside OVRCameraRig), set:
- Hand Tracking Support → “Controllers and Hands”
- Passthrough Support → “Supported”
- Enable Passthrough → Checked
Add OVRControllerPrefab and OVRHandPrefab to Right/LeftHandAnchor.
For RightHandPrefab, change to hand right in components: OVR Hand, OVR Skeleton and OVR Mesh comp.

💡 Why: The default Main Camera does not support VR tracking or Passthrough. So we have to replce the deafault Main Camera into OVRCameraRig which handels stero rendering, tracking, and device input. Changing settings in OVRManager enables both controllers and hands in MR while turing on Passthrough. OVRControllerPrefab and OVRHandPrefab provide controller/hand models so you can actually see and interact with them in MR.

2. Adding the Passthrough Layer

Create an empty object Passthrough.
Reset its Transform.
Add OVR Passthrough Layer component inside Passthrough.
Change Placement → Underlay.

💡 Why: Underlay makes the real-world view sit behind virtual objects, avoiding weird overlaps.

In CenterEyeAnchor, set Clear Flags to Solid Color and backaground to black.
Create 3D object Cube in Hierarchy and adjust the position as needed.

💡 Why: Prevents Unity’s default skybox or other background from showing through. A quick test object to confirm Passthrough + rendering are working.

Video

Scan Space

1.OVR Settings

Add OVRSceneManager.
1. Assign Plane (OVR Scene Anchor) to Plane Prefab and Volume (OVR Scene Anchor) to Volume Prefab (plane mesh & volume).
2. Assign Invisible Plane (OVR Scene Anchor) to Plane Prefab and Invisible Volume (OVR Scene Anchor) to Volume Prefab (plane mesh & volume). — recommended.
In OVRCameraRig, change to Supported in scene support.

💡 Why: Plane and Volume in OVRSceneManager enables scene understanding so your virtual objects can collide with real-world walls/floors. Invisible prefabs keep the real-world mesh hidden but functional. Without enabling scene support in OVRCameraRig, the headset won’t actually run scene scanning.

Video

Ball Interaction

1. Ball Interaction Script

Go RightHandAnchor → Add Component and create a BallInteraction component.
Copy and paste the below script.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class BallInteraction : MonoBehaviour
{
    public GameObject prefab;
    public float spawnSpeed = 5;

    void Update()
    {
        if (OVRInput.GetDown(OVRInput.Button.SecondaryHandTrigger))
        {
            GameObject Ball = Instantiate(prefab, transform.position, Quaternion.identity);
            Rigidbody BallRB = Ball.GetComponent<Rigidbody>();
            BallRB.velocity = transform.forward * spawnSpeed;
        }
    }
}

💡 Why: This script lets you spawn and throw balls from your right hand trigger.

2. Physics

Create a Physic Material Bounce (Bounciness=1) in Assets folder.
Create a 3D object sphere Bounce Sphere (scale 0.1, 0.1, 0.1) and add Rigidbody.
Go Bounce Sphere → Sphere Collider → Material and assign Bounce.
Add Bounce Sphere in Asset folder and remove it in Sample Scene.
Go RightHandAnchor → Ball Interaction → Prefab and assign Bounce Sphere.

💡 Why: The Rigidbody and Physics Material make the ball actually bounce instead of rolling lifelessly, and saving it as a prefab keeps spawning clean and reusable.

Go Edit → Project Settings → Time and set Fixed TImeStep = 0.0083333.

💡 Why: Matching Unity’s physics update to 90 Hz reduces jitter in MR, and linking the prefab ensures that pressing the trigger spawns bouncy balls.

Video

Errors

AirLink → blurry visuals, slower loads.
USB builds → load more than 10 mins sometimes.
Unity sometimes didn’t detect Quest if Wi-Fi was off or connected to different Wi-Fi with your computer.

Future Usage

Interactive Navigation Cues: Seeing balls collide with scanned walls made me think: if objects can interact reliably with meshes, you could use the same logic to overlay arrows indoors.
Spatial Data Visualization: Aligning balls with real surfaces highlighted the importance of accurate mesh placement. This directly extends to real-time overlays on equipment.
Room-Aware Game Mechanics: Physics interactions with walls suggest gameplay mechanics: your real room can become the level itself.
Furniture / Object Placement: Adjusting spawn positions to avoid clipping with walls/furniture revealed the potential to preview objects realistically in your own space.

ROS2 & Unity TCP Tutorial

Thu, 24 Jul 2025 00:00:00 GMT

ROS TCP

ROS2 is often seen as the robot’s brain, while Unity serves as the eyes and environment. To simulate realistic robotics behavior, we need a bridge between them. That bridge is the ROS-TCP Endpoint/Connector. By connecting both, we can test navigation, visualization, and SLAM in a controlled environment before deploying to real hardware.

Understanding the Basics

ROS TCP Endpoint/Connector

Def: A ROS node that enables message passing over TCP between ROS2 and external systems like Unity
Purpose: : Facilitates real-time communication for robotics simulations and visualizations
Example: A ROS node sends robot position data to Unity, which renders the robot in a 3D environment

ROS-Unity Communication

Image Source: MDPI

Def: The broader process of how ROS 2 and Unity interact, using the TCP Endpoint/Connector to exchange data via topics, services, or actions.
Example: Unity subscribes to a ROS topic (/robot_color) to change a robot’s color, or ROS responds to a Unity service request with a robot’s pose.

SLAM (Simultaneous Localization & Mapping)

Image Source: LASER_Scanning and KODIFLY

Def: A method for robots to build a map of an unknown environment while tracking their location within it.
Applications: Autonomous navigation, 3D map reconstruction, obstacle avoidance.
Example: A robot uses LIDAR to create a 2D occupancy grid in RViz, visualized in Unity.

Materials

Github

ROS TCP Endpoint

ROS TCP Connector
ROS (I used Ubuntu 22.04 with ROS2 Humble)
Unity v.2020.2+
Visual Studio (for C#)
RViz (for visualization in ROS)

Tutorial

Since I used ROS2, this tutorial is written based on ROS2 (especially ROS2 Humble).

Setup

ROS

Download ROS TCP Endpoint
In your src folder of ROS TCP Endpoint, navigate to your Colcon worspace and run the following commands:

source install/setup.bash
colcon build
source install/setup.bash

In your Colcon workspace, run the following command, replacing <your IP address> with your ROS machine's IP or hostname.

ros2 run ros_tcp_endpoint default_server_endpoint --ros-args -p ROS_IP:=192.168.0.5

If you're running ROS in a Docker container, 0.0.0.0 is a valid incoming address, so you can write ros2 run ros_tcp_endpoint default_server_endpoint --ros-args -p ROS_IP:=0.0.0.0
On Linux you can find out your IP address with the command hostname -I
On MacOS you can find out your IP address with ipconfig getifaddr en0

Once the server_endpoint has started, it will print something similar to [INFO] [1603488341.950794]: Starting server on 192.168.50.149:10000. (Alternative) If you need the server to listen on a port that's different from the default 10000, here's the command line to also set the ROS_TCP_PORT parameter:

ros2 run ros_tcp_endpoint default_server_endpoint --ros-args -p ROS_IP:=127.0.0.1 -p ROS_TCP_PORT:=10000

Unity

Open the Package Manager (Window > Package Manager)
Click the + button in the upper lefthand corner of the window. Select Add package from git URL...
Enter the git URL for the package.
For the ROS-TCP-Connector, enter https://github.com/Unity-Technologies/ROS-TCP-Connector.git?path=/com.unity.robotics.ros-tcp-connector.

(not necessary)For Visualizations, enter https://github.com/Unity-Technologies/ROS-TCP-Connector.git?path=/com.unity.robotics.visualizations.

Click Add.
open Robotics/ROS Settings from the Unity menu bar, and set the ROS IP Address variable to the IP you set earlier. (If you're using Docker, leave it as the default 127.0.0.1.)
in the ROS Settings window, ROS2 users should switch the protocol to ROS2 now.

ROS Unity Integration

Publisher

Unity publishes data (e.g., robot position coordinates) to a ROS 2 topic, which a ROS subscriber receives.

Concept: Unity can act as a Publisher, sending data continuously to a ROS topic. Typical Use Case: Streaming an object’s position, velocity, or any sensor-like data from Unity to ROS. Unity Side: You use the ROSConnection component to define the topic and message type, then call Publish() inside a Unity script. ROS Side: A ROS node subscribes to the same topic to consume Unity’s data.

Subscriber

ROS2 publishes a topic to change the color, and Unity subscribes to that topic to update it.

Concept: Unity can also subscribe to ROS topics and react when messages arrive. Typical Use Case: ROS publishes sensor data or control commands, and Unity updates its simulation accordingly (e.g., changing color, triggering animations). Unity Side: In Unity, you register a callback for a topic using Subscribe<T>(). ROS Side: A ROS node publishes messages to that topic.

Service

Unity (Service Client) request an object’s pose in Unity. ROS (Service Server) calculates its pose and responds to Unity.

Concept: Unlike Publishers/Subscribers, a Service works as a request–response pattern. Typical Use Case: When Unity needs a precise answer at a specific moment. Unity as Service Client: Unity sends a request, e.g., “What’s the pose of this object?” ROS as Service Server: ROS computes the answer and returns it.

Service Call

Start at current position
Move toward destination
Near target → Request update from ROS
Get new destination
Repeat

Concept: A Service Call can be repeated in sequence to drive a process step by step. Typical Use Case: Unity moves an object toward a goal. Once near the target, Unity requests a new goal from ROS, receives it, and continues.

Note: While Publisher and Subscriber cover most real-time data flows, Services shine when you need precise, one-off information. The Service Call tutorial illustrates how even a simple request–response can orchestrate dynamic behavior when chained together.

In practice, you’ll often combine these: Publishers for streaming state, Subscribers for reacting to commands, and Services for exact queries. The key is to choose the right communication model for the task at hand.

Robotics Nav2 SLAM Example

Running

Clone the Nav2 SLAM Example and run:

git clone https://github.com/Unity-Technologies/Robotics-Nav2-SLAM-Example.git
cd Robotics-Nav2-SLAM-Example/ros2_docker/colcon_ws

➡️ Expected Output: A 2D occupancy grid in RViz showing the robot’s map, with real-time updates as it navigates. You can extend this with custom visualization (e.g., battery status, camera feeds).

Visualization & Custom Visualizer

DefaultVisualizationSuite includes:

GoalPose → Robot position (topic-based).
OccupancyGridVisualizer → SLAM map.
LaserScanSensor → LiDAR scan.

You also need to add the following code in your Unity.

using System;
using System.Collections.Generic;
using RosMessageTypes.Geometry;                     // Generated message classes
using Unity.Robotics.Visualizations;                // Visualizations
using Unity.Robotics.ROSTCPConnector.ROSGeometry;   // Coordinate space utilities
using UnityEngine;

public class PoseTrailVisualizer : HistoryDrawingVisualizer<PoseStampedMsg>
{
    [SerializeField]
    Color m_Color = Color.white;
    [SerializeField]
    float m_Thickness = 0.1f;
    [SerializeField]
    string m_Label = "";

    public override Action CreateGUI(IEnumerable<Tuple<PoseStampedMsg, MessageMetadata>> messages)
    {
        return () =>
        {
            var count = 0;
            foreach (var (message, meta) in messages)
            {
                GUILayout.Label($"Goal #{count}:");
                message.pose.GUI();
                count++;
            }
        };
    }

    public override void Draw(Drawing3d drawing, IEnumerable<Tuple<PoseStampedMsg, MessageMetadata>> messages)
    {
        var firstPass = true;
        var prevPoint = Vector3.zero;
        var color = Color.white;
        var label = "";

        foreach (var (msg, meta) in messages)
        {
            var point = msg.pose.position.From<FLU>();
            if (firstPass)
            {
                color = VisualizationUtils.SelectColor(m_Color, meta);
                label = VisualizationUtils.SelectLabel(m_Label, meta);
                firstPass = false;
            }
            else
            {
                drawing.DrawLine(prevPoint, point, color, m_Thickness);
            }

            prevPoint = point;
        }

        drawing.DrawLabel(label, prevPoint, color);
    }
}

Errors

ROS Error

Colcon Build

Wrong package name: ROS-TCP-Endpoint-main → ros_tcp_endpoint
Wrong build location: colcon build inside Robotics-Nav2-SLAM-Example → Robotics-Nav2-SLAM-Example/ros2_docker/colcon_ws

Not Connected to Unity

Wrong Wi-Fi
Wrong ROS IP Address

Unity Error

DeserializationException: Cannot deserialize message

Rebuild msg files

ROS2 Tutorial

Mon, 14 Jul 2025 00:00:00 GMT

Understanding the Basics

ROS (Robot Operating System)

ROS2 is essentially a collection of software libraries and tools designed to streamline robot development. It provides the building blocks for communication, control, and simulation so that each component of a robot can remain modular and independent. While ROS1 already established this architecture, ROS2 pushes things further by adopting DDS (Data Distribution Service), resulting in better performance, improved reliability, and true distributed communication across a network.

Node

A node is the smallest unit of computation in ROS2. Each node has a clear, single responsibility—whether that’s reading sensor data, processing images, or issuing motor commands. Nodes communicate with each other through standardized interfaces, which is what allows us to scale from a toy simulation to a complex robot without rewriting the entire system.

Topic

Topics are the primary data channels between nodes. They support 1:N and N:N communication, making them ideal for streaming sensor data or publishing continuous control commands. If multiple subscribers are listening, ROS2 handles the distribution—something that becomes essential once a system grows beyond a few components.

Parameter

Parameters are configurable values stored inside a node. They allow runtime tuning without code changes—for example, adjusting a robot’s speed limit or sensor thresholds. While they seem simple, parameters are surprisingly powerful when combined with launch files and dynamic updates.

Service

A service provides a synchronous, request–response communication pattern. Since calls are deterministic and 1:1, services work best for short operations such as resetting a simulation, querying a map, or changing a mode. Services use .srv files to define their request and response format.

Action

Actions extend the idea of services by supporting long-running tasks with continuous feedback. Navigation is a typical example: you send a goal, receive progress updates, and eventually get a result. The .action file describes this three-part structure.

Interface

ROS2 defines three file types for communication:

.msg → basic message structures
.srv → service (request/response)
.action → long-running actions with feedback

These interfaces ensure consistent communication across nodes, packages, and even different programming languages.

Launch

Launch files are one of the major conveniences in ROS2. Instead of manually running dozens of commands, you can start entire systems—nodes, parameters, remappings, and environment configuration—through a single Python-based launch file. As projects grow, launch files become essential for keeping everything organized.

Composition

Node composition lets multiple nodes run inside a single process. This significantly reduces overhead and improves performance, especially for systems with high-frequency communication. It’s one of those features that doesn’t seem necessary at first, but becomes valuable when optimizing for real robots.

Executor

Executors determine how callbacks are processed:

Single-threaded executor — each callback runs sequentially
Multi-threaded executor — callbacks may run in parallel

Understanding executors is important for writing predictable, real-time-safe behavior.

Materials

To follow most ROS2 tutorials comfortably, the following setup works best:

Ubuntu 22.04
Official ROS2 Humble Docs
YouTube video tutorials
Visual Studio Code (C++ / Python)
Additional tools:
- Turtlesim for basic concepts
- Gazebo for simulation
- RViz 2 for visualization

Materials

Ubuntu 22.04
Official ROS2 Humble Docs
Yutube Videos
Visual Studio (C++ / Python)
Additional Tools
- Turtlesim
- Gazebo (Sim)
- RViz 2

Tutorials

Beginner: CLI Tools

Before writing any code, ROS2 encourages you to explore the ecosystem through the command line. This stage is surprisingly valuable because it forces you to understand how nodes, topics, services, and actions behave in a real running system.

What you learn here

Inspecting active nodes with ros2 node list
Checking published and subscribed topics with ros2 topic list
Echoing data streams using ros2 topic echo
Calling services directly through CLI
Visualizing architecture using rqt_graph
Basic debugging with ros2 doctor and ros2 info

Most developers (including me) underestimate this step, but these CLI tools become indispensable once you’re debugging large multi-node systems.

Beginner: Client Libraries (Python & C++)

Once you get familiar with how ROS2 works under the hood, the next step is building your own packages. This is where colcon and the workspace structure start making sense. You create a src/ folder, add packages, write nodes, and build everything in a clean, modular layout.

Core Topics

1. Creating Packages

You learn to generate packages using:

ros2 pkg create my_pkg --build-type ament_cmake

or Python packages via:

ros2 pkg create my_pkg --build-type ament_python

This automatically sets up the directory structure, package.xml, and build files.

2. Publisher & Subscriber

This is the “Hello World” of ROS2 development. Writing a minimal pub/sub pair teaches you:

message imports
QoS (Quality of Service) basics
callback structures
timers

It’s simple, but it becomes the foundation for almost every robotics system.

3. Services & Clients

Here you write synchronous communication in both languages. You define .srv files, build interfaces, and implement request–response handlers.

4. Custom Interfaces

You define your own .msg and .srv structures. This part forces you to think about architecture instead of dumping arbitrary data.

5. Parameters

ROS2’s parameter system allows you to:

load parameters from YAML
override them via CLI
declare dynamic parameters
handle parameter callbacks

This becomes extremely useful in launch files and tuning algorithms.

Intermediate: Advancing with ROS2

This stage covers the actual building blocks that real robots rely on. It’s where everything you learned starts connecting, and where the architecture begins to feel more intentional.

Working With Dependencies

Before building any ROS2 package, you need dependency resolution. rosdep handles this by scanning package.xml and installing missing system libs.

rosdep install --from-paths src -y --ignore-src

It looks like a minor detail, but forgetting this step is responsible for 80% of beginner build failures.

Actions

Actions take services to the next level. You define .action files with:

Goal
Result
Feedback

Then you implement action servers and clients in both Python and C++. Navigation stacks and manipulation frameworks rely heavily on actions.

Composable Nodes

One of the most powerful features in ROS2. Instead of launching each node as a separate process, you load multiple nodes into a single shared process.

Why it matters:

lower latency (no process-to-process overhead)
reduced memory consumption
faster communication

Composable nodes make a huge difference in real robot performance.

Node Interfaces & Dynamic Parameters

This covers more advanced techniques like:

introspecting node interfaces
reacting to parameter updates
using templates and advanced C++ patterns

It helps you understand how nodes should behave in large-scale systems.

Launch Files (Python / XML)

Launch files become more complex here:

multiple nodes
namespaces
remapping
events and lifecycle management
conditional execution
passing parameters to nodes automatically

You essentially design how your entire robotics system boots up.

TF2 (Transforms)

TF2 is the coordinate transform system of ROS2. Every robot with physical movement uses it.

You learn:

broadcasting transforms
listening and chaining transforms
using TF buffers
visualizing frames in RViz

Understanding TF2 is crucial for anything involving sensors or movement.

Testing (Unit & Integration)

ROS2 has built-in testing through:

GTest for C++
pytest for Python
launch_testing for integration tests

Testing robotic systems might feel optional, but once you run a multi-node stack, automated tests save hours of debugging.

Robot Modeling (URDF) & Visualization

URDF (Unified Robot Description Format) is used to model robot links and joints. You visualize it in RViz and integrate it with Gazebo simulation.

This is the point where your ROS2 knowledge becomes tangible—you can visualize your own robot moving, even before a physical prototype exists.

Debugging Notes

Some common issues that appear while building ROS2 projects:

Missing dependencies (fix with rosdep install)
Incorrect package paths
Misconfigured CMakeLists or package.xml
Build failures due to unsaved files (surprisingly common)
Gazebo version conflicts
RViz plugin issues
Wrong QoS causing subscriber to receive no messages

Debugging ROS2 is an ongoing skill, not a one-time lesson.

VR Game Tutorial

Mon, 07 Jul 2025 00:00:00 GMT

Introduction

I recently followed Valem’s “How to Make a VR Game in Unity” series (13 videos) to understand the fundamentals of VR interaction using the Unity XR Interaction Toolkit. The videos cover everything from basic VR setup to locomotion, grabbing, haptics, UI interaction, and comfort features. While the tutorial itself is straightforward, implementing it in a real Unity project exposed a lot of small details that matter more than expected. This post is a structured summary of everything I learned along the way—both from the guide and from debugging my own mistakes.

Tutorial Overview

Source: “How to make a VR game in Unity” by Valem (13-video series)
Hardware: Meta Quest 2 / Quest Link
Software: Unity 2023.x + XR Interaction Toolkit

Video Breakdown

1–2. VR Overview & Unity Setup
3. Hand Models & Input
4. Continuous Move & Turn
5. Teleportation
6. Hover / Grab / Interactables
7. Offset Grab & Distance Grab
8. VR UI
9. Haptics
10–11. Grab Poses
12. Tunneling Vignette (comfort)
13. Example Project Overview

Development Steps

Programming (C#)

Throughout the tutorial I implemented several core interaction scripts:

custom input detection (pinch, grip, trigger) using the Input System
projectile / bullet firing logic
vibrating controllers via XR haptics
enforcing custom grab poses
automatically disabling teleport rays while interacting with UI
managing interaction layers for direct vs. ray interactors

Implementing these by hand gave me a much better understanding of how XR Interaction Toolkit actually processes events behind the scenes.

Unity & XR Interaction Toolkit

Core Unity steps included:

enabling OpenXR in XR Plugin Management
setting up the XR Origin (camera + hands)
using XR Ray Interactor for teleportation and distance grabbing
building interactive World Space UIs
adding continuous locomotion and snap turning
configuring Tunneling Vignette to reduce motion sickness
mirroring hand poses between left and right hands
organizing interaction layers

Even though Unity automates a lot of VR setup today, the small configuration details still matter—especially when multiple systems like locomotion, UI, grabbing, and physics overlap.

Step-by-Step Implementation

Video 1–2: VR Overview & Setup

1. Project Setup

Created a new Unity project
Adjusted project settings (rendering, physics, XR defaults)
Added a main VR scene with a plane and basic environment

2. XR Setup

Installed XR Interaction Toolkit via Package Manager
Added XR Origin containing camera and both hand controllers
Configured Input Actions for both controllers

3. XR Plugin

Enabled OpenXR
Activated required interaction profiles

This part seems simple, but OpenXR configuration must match the headset—otherwise nothing responds.

Video 3: Hand Models & VR Input

Hand Setup

Imported custom hand models
Placed them under LeftHand / RightHand
(Optional) Added Animator controllers for finger motions

Input Detection

Coded detection for:

trigger
grip
pinch

This made it easier to connect interactions like firing bullets or grabbing interactables.

Video 4: Continuous Movement & Turning

Continuous Locomotion

Implemented using:

Continuous Move Provider
Character Controller
Gravity + center adjustments

Turn Provider

Two modes:

Continuous Turn (smooth rotation)
Snap Turn (fixed-angle rotation for comfort)

Movement feels simple, but balancing gravity, character height, and collision takes trial and error.

Video 5: Teleportation

Teleportation Setup

Added XR Ray Interactor to each controller
Configured straight or curved ray
Added Teleportation Provider
Defined teleportation areas (floors, platforms)
Added a hover reticle for valid targets

Teleport is still the most comfortable VR locomotion option, so getting this right matters.

Video 6: Hover, Grab, and Interactables

Hover Detection

Used XR Direct Interactor + Collider.

Grab Interactables

Configured:

Rigidbody
Collider
Interaction settings (instantaneous / velocity-based / kinematic)

Projectile Firing

Imported a pistol model and implemented forward bullet firing.

This was the first part that made the project feel like an actual VR game.

Video 7: Offset Grab & Distance Grab

Offset Grab

Created attachment transforms for left and right hands so each object is held naturally.

Distance Grab

Enabled remote grabbing using:

XR Ray Interactor
Interaction layers

Layer management is surprisingly easy to overlook, but it's crucial for preventing your rays from grabbing UI or teleport surfaces unintentionally.

Video 8: VR UI

World-Space UI

Added a World Space canvas
Attached buttons, sliders, toggles
Set UI layer + interaction components

Handling Interaction Conflicts

Disabled teleport rays whenever hands interacted with UI—preventing accidental teleports.

Dynamic UI

Added a toggle panel that appears in front of the user.

Video 9: Haptic Feedback

Haptic Setup

Attached custom haptic scripts
Triggered haptics on grab/release
Controlled intensity + duration

Haptics are a small detail but dramatically improve interaction feel.

Video 10–11: Grab Poses

Custom Grab Poses

Captured hand joint data for right hand
Applied pose to interactable via attach transform
Mirrored pose to left hand

Custom grab poses make objects feel much more realistic, especially tools or weapons.

Video 12: Tunneling Vignette

Comfort Feature

Imported Tunneling Vignette sample
Configured feathering, fade times, colors
Attached component to main camera

This reduces VR motion sickness by limiting peripheral vision during movement.

Video 13: Example Project

Explored Unity’s built-in XR samples:

interactables
physics interaction
socket interactables
climbing
gaze input
locomotion
both 2D & 3D UI

This was helpful to understand more complex, real-world setups.

Problems I Encountered

1. Unity Version Mismatch

Issue: Certain XR features didn't match the tutorial.
Fix: Downgraded Unity to 2023.3.16f1 (matching the tutorial’s version).

2. Meta Quest Link Issues

Issue: Quest Link displayed in low resolution or failed to connect.
Fix: Switched PCs + updated GPU drivers & Oculus app.

3. Input Misconfiguration

Issue: Trigger / grip inputs not firing correctly.
Fix: Rewrote entire Input Action asset and fixed bindings.

Debugging VR setups often feels like fighting invisible settings—especially with the Input System.

Questions Going Forward

Following the tutorial clarified the basics, but it also left me with a few questions that matter for real development:

Should I rely on Unity’s default grab system or fully customize grab poses per object?
Are there recommended architecture patterns for complex VR interactions?
How can I simulate and debug VR features quickly without wearing the headset every time?

These are areas I want to explore more deeply as I start developing my own VR mechanics.

Eunha

XR Interaction Toolkit (Unity VR Pathway)

Materials

Basic Setting

With the Device Simulator

C# Studying

C# Studying

1. Variables and Data Types

What is a variable?

Value Types

Variable Initialization

2. Operators

Arithmetic Operators

Division and Casting

3. Conditional Statements

if / else

Logical Operators

switch and enum

4. Loops

for loop

while / do-while

break and continue

5. Arrays

Why arrays exist

2D Arrays

6. Methods (Functions)

Why methods matter

Example

MCP with Blender & Unity

MCP in Blender & Unity with Claude AI

Understanding MCP

Tools & Sources

Step-by-Step Setup

Setup

1. UV Package Manager

Blender MCP Setup

1. Addon Installation

2. Blender MCP Tab

3. Claude AI Config

Video

Unity MCP Setup

1. Python Path Setup (Windows)

2. Package Installation

3. MCP Auto-Setup

Video

Discussion & Reflections

Future Outlook

Blender Tutorial

My Blender Tutorial Experience

Understanding the Basics

Tools & Resources

Essential Shortcuts

Recommended Setup

Step-by-Step Experience

Tutorial 1: Blender Beginner Tutorial

1. Cookie Base

2. Chocolate Chips

3. Tray

4. Materials & Colors

5. Lighting & Rendering

Tutorial 2: Blender Floor Plan Tutorial

1. Setup

2. Floor & Walls

3. Finalizing & Rendering

Summary

MR Meta Quest Step-by-Step Tutorial

Mixed Reality on Meta Quest 3

Understanding the Basics

Tools

Step-by-Step Tutorials

Setup

1. Enable Developer Mode

2. Setting Up Unity for Oculus Development

Connecting the Headset & Building

Method 1: AirLink (Wireless)

Method 2 (Recommended): USB-C (Wired)

Passthrough

1. Camera Configuration

2. Adding the Passthrough Layer

Video