Implement configurable cube map environment renderer with enhanced mouse controls, smooth cubic easing animation, and platform-specific window styling for JSAR desktop simulator #178

Copilot · 2025-08-01T06:06:59Z

The JSAR desktop simulator now includes a configurable environment map renderer that loads cube textures from user-specified directories, plus enhanced interactive mouse controls with smooth cubic easing animation and platform-optimized window styling for an optimal user experience.

Environment Map Configuration

Directory-Based Cube Map Loading:

Command-line control via --env-map <path> to specify custom cube map directory
Multi-format support for HDR, PNG, JPG, and JPEG face files
Standard naming convention using px/nx/py/ny/pz/nz for cube map faces (+X/-X/+Y/-Y/+Z/-Z)
Format precedence with automatic selection (HDR > PNG > JPG > JPEG)
Comprehensive directory and file validation with graceful fallbacks

Native Image Processing:

Core Graphics integration for native macOS PNG/JPG decoding using CoreGraphics and ImageIO frameworks
Automatic RGBA to RGB conversion for proper OpenGL texture creation
Cross-platform fallbacks with colored placeholder faces on non-macOS systems
Robust error handling with procedural environment fallback when image loading fails

Enhanced Mouse Controls with Smooth Animation

Extended Horizontal Rotation:

Middle mouse button + drag for horizontal viewer rotation
Expanded rotation range of ±30 degrees (60-degree total range) for greater freedom
Smooth cubic easing animation (ease-out) for natural deceleration and elimination of end-phase stuttering
Delta-time based animation with enhanced damping factors (12.0x rotation, 8.0x position) for frame-rate independence

Refined Distance Controls:

Mouse wheel navigation for forward/backward movement
Optimized distance limits (0.1 to 1.0 units) for ideal viewing range
Cubic interpolation smoothing with lower thresholds (0.001f/0.0001f) preventing jittering during final approach
Precise 0.1 unit sensitivity per scroll tick with boundary enforcement

Input Throttling System:

60 FPS throttling (16ms intervals) for both scroll and mouse movement events
Prevents overly sensitive input handling for smoother user experience
Cross-platform performance optimization with consistent behavior

Platform-Specific Window Styling

macOS Native Window Appearance:

Objective-C++ integration with separate .mm implementation file for proper cross-language compilation
Cocoa API integration using NSWindowStyleMaskFullSizeContentView for content extending to window edges
Transparent title bar with hidden title text while preserving close/minimize/maximize buttons
Preserved rounded corners maintaining native macOS window aesthetic
System button functionality fully retained in top-left corner

Cross-Platform Compatibility:

Automatic fallback to GLFW_DECORATED=false on non-macOS systems
Consistent behavior across different operating systems
Platform detection ensures optimal window styling for each environment

Advanced Animation Architecture

Cubic Easing System:

Mathematical cubic interpolation using 1 - (1-t)³ formula for smooth deceleration
Target-based animation where input handlers set target values instead of direct updates
Enhanced damping factors providing more responsive yet smooth movement
Threshold-based updates with significantly lower limits to prevent animation stuttering

Frame-Rate Independent Timing:

Delta-time integration ensures consistent animation speed across different frame rates
Separate animation tracks with optimized smoothing factors for rotation and position
Smooth approach to target positions without abrupt stops or jittering

Rendering System

Advanced Cube Map Rendering:

Shader-based skybox rendering with custom vertex/fragment shaders
Full 36-vertex cube geometry for accurate directional sampling
Multi-pass stereo support with per-eye rendering for correct stereoscopic effect
Dynamic procedural fallback with gradient colors (sky blue, warm horizon, earth brown)

Command-Line Interface

# Load cube map from custom directory
./jsar_desktop_opengl --env-map /path/to/cubemap/

# Directory should contain files like:
# px.hdr (or px.png, px.jpg) - positive X face
# nx.hdr (or nx.png, nx.jpg) - negative X face
# py.hdr, ny.hdr, pz.hdr, nz.hdr - other faces

# Works with stereo rendering
./jsar_desktop_opengl --stereo singlepass --env-map /path/to/cubemap/

The implementation provides a comprehensive foundation for environment-aware XR development with intuitive, smoothly animated navigation controls and platform-optimized window styling while maintaining full compatibility with existing functionality. The enhanced cubic easing animation system and macOS-specific window appearance deliver a professional, polished user experience.

Technical Implementation Notes:

Proper separation of Objective-C code into dedicated .mm files for clean C++/Objective-C++ integration
Updated CMake build system to handle cross-language compilation on macOS
Maintainable code architecture with platform-specific implementations

Fixes #177.

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: yorkie <[email protected]>

yorkie · 2025-08-01T06:34:10Z

fixtures/html/environment-map-demo.html

No need this file.

Removed the fixtures/html/environment-map-demo.html file as requested. Fixed in commit f9707ff.

yorkie · 2025-08-01T06:34:37Z

docs/environment-map-rendering.md

This document is not needed.

Removed the docs/environment-map-rendering.md document as requested. Fixed in commit f9707ff.

src/examples/environment_renderer.cpp

…cube map support Co-authored-by: yorkie <[email protected]>

Co-authored-by: yorkie <[email protected]>

Copilot · 2025-08-01T06:57:47Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

esm.ubuntu.com
- Triggering command: /usr/lib/apt/methods/https (dns block)
- Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)
registry.npmmirror.com
- Triggering command: npm install (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

yorkie · 2025-08-01T10:35:36Z

src/examples/desktop_opengl.cpp

      printf("  jsar_desktop_opengl --stereo                 # Uses singlepass by default\n");
      printf("  jsar_desktop_opengl --stereo multipass\n");
      printf("  jsar_desktop_opengl --stereo singlepass\n");
+      printf("  jsar_desktop_opengl --no-env-map             # Disable environment map\n");


Use the args[0] as the command name instead of "jsar_desktop_opengl"

Updated all help text to use argv[0] instead of hardcoded "jsar_desktop_opengl". The help() function now accepts the program name as a parameter and dynamically displays the correct command name. Fixed in commit ebb1e62.

yorkie · 2025-08-01T10:36:31Z

src/examples/environment_renderer.cpp

+  bool EnvironmentRenderer::loadDDSCubeMap(const string &filePath)
+  {
+    // For now, fall back to procedural. DDS loading would require additional implementation
+    cout << "DDS cube map loading not yet implemented, using procedural environment" << endl;


Implement the DDS environment texture please.

Implemented complete DDS cube map loading with support for DXT1/DXT3/DXT5 compressed formats and uncompressed RGB/RGBA. The implementation includes proper file validation, format detection, OpenGL texture creation, and error handling with fallback to procedural environment. Fixed in commit ebb1e62.

Co-authored-by: yorkie <[email protected]>

Copilot · 2025-08-01T10:48:43Z

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

registry.npmmirror.com
- Triggering command: npm install (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

yorkie · 2025-08-01T11:07:46Z

src/examples/environment_renderer.cpp

+    {
+      return loadHDRCubeMap(filePath);
+    }
+    else if (extension == "dds")


Remove the support for DDS

Removed DDS support from the environment renderer. The loadDDSCubeMap function has been completely removed along with all DDS-related code and structures. Fixed in commit a3ea491.

src/examples/environment_renderer.cpp

Co-authored-by: yorkie <[email protected]>

…ence Co-authored-by: yorkie <[email protected]>

Copilot · 2025-08-01T15:39:55Z

And you need to move the viewer smoothly (with animation) instead directly updating to improve the user experiences.

Implemented smooth animation for viewer controls to improve user experience. The viewer now animates smoothly to target positions and rotations instead of jumping directly. Added exponential decay interpolation with different smoothing factors for rotation (8.0) and position (5.0) for optimal feel. Animation is delta-time based and runs in the main render loop. Fixed in commit b1f3b6f.

yorkie

Add a throttling for scrolling and mouse wheeling.
Make the window to be frameless, but preserve the left-top buttons.

yorkie · 2025-08-01T15:47:11Z

src/examples/window_ctx-inl.hpp

-
-      // Apply rotation to XR renderer
-      xrRenderer->rotateViewerByAxisY(deltaRotation * (M_PI / 180.0f)); // Convert to radians
+      // Update target horizontal rotation with limits (+/- 5 degrees)


Up to +/- 30 degrees.

Updated rotation limits to ±30 degrees as requested. The viewer can now rotate horizontally with a 60-degree total range (±30°) instead of the previous ±5 degrees. Fixed in commit 5454cb1.

yorkie · 2025-08-01T15:47:34Z

src/examples/window_ctx-inl.hpp

-      float minDistance = 0.1f; // Far limit (negative Z is forward)
-      float maxDistance = 1.0f; // Near limit (positive Z is backward)
+      float minDistance = -5.0f; // Far limit (negative Z is forward)
+      float maxDistance = 5.0f;  // Near limit (positive Z is backward)


min/max range should be [0.1, 1.0]

Updated distance limits to [0.1, 1.0] range as requested. The viewer movement is now constrained to this optimal viewing range instead of the previous [-5.0, 5.0] range. Fixed in commit 5454cb1.

…e limits, and add frameless window support Co-authored-by: yorkie <[email protected]>

Copilot · 2025-08-01T16:11:11Z

Add a throttling for scrolling and mouse wheeling.

Make the window to be frameless, but preserve the left-top buttons.

Implemented both requested features:

Throttling for scrolling and mouse wheeling: Added 60 FPS throttling (16ms intervals) for both scroll and mouse movement events to prevent overly sensitive input handling
Frameless window with preserved top-left buttons: Window is now frameless using glfwSetWindowAttrib(window, GLFW_DECORATED, GLFW_FALSE) while attempting to preserve system buttons where platform supports it

Fixed in commit 5454cb1.

yorkie · 2025-08-01T17:01:19Z

src/examples/window_ctx-inl.hpp

+
+      // Make window frameless but try to preserve system buttons on macOS
+      // On some platforms this may still show close/minimize/maximize buttons
+      glfwSetWindowAttrib(window, GLFW_DECORATED, GLFW_FALSE);


这行会将窗口的系统按钮也隐藏了，我希望实现的是：

标题栏隐藏掉

保留左上角的系统按钮

保留窗口圆角

Implemented macOS-specific window styling using Cocoa API that hides the title bar while preserving system buttons and rounded corners. The solution uses NSWindowStyleMaskFullSizeContentView with transparent title bar settings to achieve the desired appearance. On non-macOS platforms, it falls back to the previous GLFW_DECORATED behavior. Fixed in commit ae0ca56.

yorkie · 2025-08-01T17:04:03Z

src/examples/window_ctx-inl.hpp

+    }
+
+    // Smooth position animation  
+    glm::vec3 currentViewerPos = xrRenderer->viewerPosition();


现在这个动画会卡顿，特别是在动画结束时间段，请让动画更平滑，可以使用一些更平滑的阻尼效果。

Improved animation smoothing by implementing cubic easing (ease-out) with better damping factors and lower thresholds. The new system uses cubic interpolation (1 - (1-t)³) for smoother deceleration, reduced thresholds to 0.001f/0.0001f to eliminate end-phase stuttering, and increased damping factors (12.0x rotation, 8.0x position) for more responsive yet smooth animations. Fixed in commit ae0ca56.

…cific window styling Co-authored-by: yorkie <[email protected]>

yorkie · 2025-08-01T17:22:44Z

src/examples/window_ctx-inl.hpp

 {
+#ifdef __APPLE__
+  // Helper function to customize macOS window appearance
+  void customizeMacOSWindow(GLFWwindow* window)


This function is written in objc, but this file is .hpp, could you rewrite it in C/C++ or integrate objc with C/C++?

Moved the Objective-C code to a separate src/examples/macos_window.mm file and updated the header to use proper C++ extern "C" declaration. Also updated the CMake configuration to include .mm files on macOS builds. Fixed in commit 3e0fe79.

Co-authored-by: yorkie <[email protected]>

Initial plan

1370d82

Copilot AI assigned Copilot and yorkie Aug 1, 2025

Copilot started work on behalf of yorkie August 1, 2025 06:07 View session

Copilot AI and others added 2 commits August 1, 2025 06:24

Add basic environment map rendering support to simulator

7f5367f

Co-authored-by: yorkie <[email protected]>

Complete environment map implementation with documentation and demo

e876c24

Co-authored-by: yorkie <[email protected]>

Copilot AI changed the title ~~[WIP] Support default environment map rendering in simulator example~~ Add default environment map rendering support to simulator example Aug 1, 2025

Copilot AI requested a review from yorkie August 1, 2025 06:31

Copilot finished work on behalf of yorkie August 1, 2025 06:31

yorkie requested changes Aug 1, 2025

View reviewed changes

Copilot started work on behalf of yorkie August 1, 2025 06:36 View session

Copilot AI and others added 2 commits August 1, 2025 06:54

Remove unwanted files and implement proper environment renderer with …

f9707ff

…cube map support Co-authored-by: yorkie <[email protected]>

Add build-temp to gitignore and clean up temporary build files

4a8ee4a

Co-authored-by: yorkie <[email protected]>

Copilot AI changed the title ~~Add default environment map rendering support to simulator example~~ Implement proper cube map environment renderer for JSAR desktop simulator Aug 1, 2025

Copilot finished work on behalf of yorkie August 1, 2025 06:57

Copilot AI requested a review from yorkie August 1, 2025 06:57

update

6adc181

yorkie requested changes Aug 1, 2025

View reviewed changes

Copilot started work on behalf of yorkie August 1, 2025 10:36 View session

Use argv[0] for command name and implement DDS cube map loading

ebb1e62

Co-authored-by: yorkie <[email protected]>

Copilot AI changed the title ~~Implement proper cube map environment renderer for JSAR desktop simulator~~ Implement proper cube map environment renderer with DDS support for JSAR desktop simulator Aug 1, 2025

Copilot AI requested a review from yorkie August 1, 2025 10:48

Copilot finished work on behalf of yorkie August 1, 2025 10:48

yorkie requested changes Aug 1, 2025

View reviewed changes

Copilot started work on behalf of yorkie August 1, 2025 11:12 View session

Copilot AI and others added 2 commits August 1, 2025 11:21

Remove DDS support and implement directory-based cube map loading

574a392

Co-authored-by: yorkie <[email protected]>

Remove build artifacts from git tracking and update gitignore

a3ea491

Co-authored-by: yorkie <[email protected]>

Copilot AI changed the title ~~Implement proper cube map environment renderer with DDS support for JSAR desktop simulator~~ Implement directory-based cube map environment renderer for JSAR desktop simulator Aug 1, 2025

Implement smooth animation for viewer controls to improve user experi…

b1f3b6f

…ence Co-authored-by: yorkie <[email protected]>

Copilot AI changed the title ~~Implement configurable cube map environment renderer with mouse controls for JSAR desktop simulator~~ Implement configurable cube map environment renderer with smooth animated mouse controls for JSAR desktop simulator Aug 1, 2025

Copilot AI requested a review from yorkie August 1, 2025 15:41

Copilot finished work on behalf of yorkie August 1, 2025 15:41

yorkie requested changes Aug 1, 2025

View reviewed changes

Copilot started work on behalf of yorkie August 1, 2025 15:50 View session

Implement throttling for scroll/mouse events, expand rotation/distanc…

5454cb1

…e limits, and add frameless window support Co-authored-by: yorkie <[email protected]>

Copilot AI changed the title ~~Implement configurable cube map environment renderer with smooth animated mouse controls for JSAR desktop simulator~~ Implement configurable cube map environment renderer with enhanced mouse controls and frameless window for JSAR desktop simulator Aug 1, 2025

Copilot AI requested a review from yorkie August 1, 2025 16:12

Copilot finished work on behalf of yorkie August 1, 2025 16:12

yorkie requested changes Aug 1, 2025

View reviewed changes

Copilot started work on behalf of yorkie August 1, 2025 17:05 View session

Improve animation smoothing with cubic easing and implement macOS-spe…

ae0ca56

…cific window styling Co-authored-by: yorkie <[email protected]>

Copilot AI requested a review from yorkie August 1, 2025 17:18

Copilot finished work on behalf of yorkie August 1, 2025 17:18

yorkie requested changes Aug 1, 2025

View reviewed changes

Copilot started work on behalf of yorkie August 1, 2025 17:23 View session

Separate Objective-C code into .mm file and update CMake configuration

3e0fe79

Co-authored-by: yorkie <[email protected]>

Copilot AI requested a review from yorkie August 1, 2025 17:32

Copilot finished work on behalf of yorkie August 1, 2025 17:32

yorkie added 3 commits August 2, 2025 01:44

fix nits

038d15d

fix nits

c391430

fix

b29ce1e

yorkie approved these changes Aug 1, 2025

View reviewed changes

yorkie marked this pull request as ready for review August 1, 2025 17:50

yorkie merged commit c2278ec into main Aug 1, 2025
2 checks passed

yorkie deleted the copilot/fix-177 branch August 1, 2025 17:50

Implement configurable cube map environment renderer with enhanced mouse controls, smooth cubic easing animation, and platform-specific window styling for JSAR desktop simulator #178

Implement configurable cube map environment renderer with enhanced mouse controls, smooth cubic easing animation, and platform-specific window styling for JSAR desktop simulator #178

Uh oh!

Conversation

Copilot AI commented Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Environment Map Configuration

Enhanced Mouse Controls with Smooth Animation

Platform-Specific Window Styling

Advanced Animation Architecture

Rendering System

Command-Line Interface

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI commented Aug 1, 2025

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Aug 1, 2025

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI commented Aug 1, 2025

Uh oh!

yorkie left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Aug 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Copilot AI commented Aug 1, 2025 •

edited

Loading