2023 • Custom OpenAI Gym Environment • Published in PhD Thesis Chapter 6
I developed this simple environment where a ball falls and bounces, obeying the free fall of gravity. The floor position is not included in the observation space. Can the model learn to predict the next state of this environment, for example, the bouncing of the ball when it hits the ground?
Watch the environment in action: hidden floor challenge demonstrated
When transitioning from discrete to continuous action spaces in SocNavGym, I observed that as action choices increased from 4 to 16, prediction errors became more pronounced. This highlighted the need for more robust sequential learning models that could handle incomplete state information.
I developed FallingBallEnv as a controlled testing ground to explore fundamental challenges in temporal prediction before applying methods to complex environments like SocNavGym. It isolates the core challenge of learning from incomplete observations, allowing for focused investigation of sequential learning capabilities without the complexity of social navigation dynamics.
To study the impact of data distribution on model performance, I created three distinct datasets:
The balanced dataset ensured models were exposed to bounce events more frequently, improving their ability to learn bounce dynamics effectively.
In FallingBallEnv, the floor position is deliberately hidden from the observation space (see Key Innovation section). Models only receive:
The challenge: Can your model learn to infer where the floor is from the ball’s trajectory alone?
This tests whether models can:
Unlike most Gym environments that reveal all state information, FallingBallEnv deliberately omits the floor position from the observation space. This creates a controlled test of sequential learning where models must infer boundary conditions from trajectory patterns alone.
Key Design Elements:
The balanced dataset (equal representation of bounce and free-fall windows) resulted in a 25% reduction in MAE compared to the unbalanced dataset, highlighting the importance of balanced training data for learning transition events like bounces.
| Model | MAE (k=1) | MAE (k=3) | MAE (k=5) | MAE (k=10) | Time (s) | Params |
|---|---|---|---|---|---|---|
| Vanilla LSTM | 1.643×10⁻⁴ | 1.705×10⁻⁴ | 1.730×10⁻⁴ | 4.696×10⁻³ | 0.00108 | 84,419 |
| GRU | 1.737×10⁻⁴ | 1.677×10⁻⁴ | 2.114×10⁻⁴ | 4.793×10⁻³ | 0.00091 | 63,363 |
| Transformer | 9.138×10⁻⁵ | 7.189×10⁻⁵ | 8.910×10⁻⁵ | 1.528×10⁻³ | 0.01417 | 1,125,059 |
| CGLSTM | 3.558×10⁻⁵ | 4.302×10⁻⁵ | 2.341×10⁻⁵ | 1.360×10⁻³ | 0.00325 | 104,771 |
Note: MAE = Mean Absolute Error, k = prediction horizon (steps ahead). Lower is better. Results from balanced dataset.
CGLSTM Achievements:
Figure 1: Environment Visualization
Ball falling and bouncing in FallingBallEnv. Yellow circle = current state. The floor position is not included in the observation space.
Figure 2: Prediction Comparison
Sky blue = actual future, blue = CGLSTM prediction, red = errors, gray = perfect alignment.
Figure 3: CGLSTM Performance
Detailed results showing CGLSTM's superior accuracy across prediction horizons (k=1, 3, 5, 10).
[x, y, vx, vy]x: Horizontal position (continuous)y: Vertical position (continuous)vx: Horizontal velocity (continuous)vy: Vertical velocity (continuous)y_floor) is deliberately excluded from observations (see Key Innovation section)# Clone the repository
git clone https://github.com/goodluckoguzie/falling_ball_env.git
cd falling_ball_env
# Install dependencies
pip install -e .
# or
pip install gymnasium numpy opencv-python
import gymnasium as gym
from falling_ball_env import FallingBallEnv
# Create environment
env = FallingBallEnv()
# Reset environment
obs, info = env.reset()
print(f"Observation: {obs}") # [x, y, vx, vy]
print(f"Notice: Floor position is NOT in observations!")
print(f"Challenge: Can your model infer where the ball will bounce?")
# The model must learn the hidden floor position
# from the ball's trajectory alone
# Run episode
for step in range(100):
# For prediction task, no action needed
# Instead, predict next observation
obs, reward, terminated, truncated, info = env.step(None)
# Render visualization
env.render()
if terminated or truncated:
obs, info = env.reset()
env.close()
# Create environment with custom parameters
env = FallingBallEnv(
gravity=9.81, # Gravitational acceleration
energy_loss=0.8, # Velocity reduction after bounce (0.7-0.9)
floor_height=0.0, # Floor position (not in observation!)
initial_y_range=(5, 10), # Initial y position range
ball_radius=0.5 # Ball radius
)
Main Methods:
reset() - Reset environment, returns (observation, info)step(action) - Execute action, returns (observation, reward, terminated, truncated, info)render() - Render visualizationclose() - Close environmentObservation Format:
[x, y, vx, vy] - 4D continuous vectorFallingBallEnv provided crucial insights into sequential learning that informed the development and validation of CGLSTM. Results demonstrated that CGLSTM’s cosine similarity gating mechanism significantly improves prediction accuracy in temporal tasks, establishing a foundation for applying the model to more complex social navigation scenarios.
Key Contributions:
FallingBallEnv was important for evaluating and validating the Cosine-Gated LSTM (CGLSTM) architecture. The environment enabled:
Used In:
Related Environments:
Comparison with Other Environments: Unlike standard Gym environments (CartPole, MountainCar, etc.), FallingBallEnv deliberately omits state information to test implicit physics learning (see Key Innovation section). This makes it unique for evaluating sequential learning models that must infer hidden dynamics.
A: This design tests whether models can learn implicit physics from observations alone. Unlike most Gym environments that reveal all state information, FallingBallEnv requires models to infer hidden boundary conditions from trajectory patterns (see Key Innovation section).
A: Most Gym environments (CartPole, MountainCar, etc.) reveal all state information. FallingBallEnv uniquely tests learning of implicit physics by hiding the floor position, creating a controlled test of sequential learning capabilities (see Key Innovation section).
A: Yes! The environment is open-source and available on GitHub. It’s particularly useful for:
A: The hidden floor position forces models to learn implicit physics from trajectory observations alone. This tests whether models can infer boundary conditions without explicit state information.
A: Clone the repository, install dependencies, and check out the Quick Start example in the Installation & Usage section above.
A: Three datasets were created:
A: Vanilla LSTM, GRU, RAU, Transformer, and CGLSTM (proposed model). CGLSTM achieved the best performance across all metrics.
If you use FallingBallEnv in your research, please cite:
BibTeX:
@software{fallingballenv2023,
title = {FallingBallEnv: A Custom Gym Environment for Sequential Learning Validation},
author = {Oguzie, Goodluck},
year = {2023},
url = {https://github.com/goodluckoguzie/falling_ball_env},
note = {Used in: Oguzie, G. (2024). Cosine-Gated LSTM for Sequential Learning.
In 2024 IEEE 5th International Conference on Pattern Recognition and Machine Learning (PRML), pp. 8-15.
PhD Thesis: Enhancing Robot Social Navigation with Reinforcement Learning and Advanced Predictive Models,
Aston University, 2024, Chapter 6}
}
APA:
Oguzie, G. (2023). FallingBallEnv: A Custom Gym Environment for Sequential Learning Validation [Computer software].
GitHub. https://github.com/goodluckoguzie/falling_ball_env
Technologies: Python • OpenAI Gym • Gymnasium • NumPy • OpenCV • Machine Learning • PyTorch
Code & Repository:
Publications & Documentation:
Related Projects: