A custom Deep Q-Network estimates future value for the enabled walking, turning, rolling, braking, flying, and landing actions. The full physical state has 18 values; the network receives only the selected inputs.
18 inputs / 32 ReLU neurons / 7 enabled actions
Forward, left, and right rays intersect obstacle bounding volumes. Each returns normalized distance, obstacle type, and the reward assigned to the intersected face. Contacts use the same north/south/east/west convention; landings and ground traps use the top reward. Sensor selection affects the network, independently of laser visibility.
Batches of 24 transitions are sampled from a 10,000-entry replay buffer. Huber-loss gradients train the online network with SGD. A target network synchronizes every 100 updates. The Bellman target is r + 0.97 max Q(s', a'), over enabled actions only; terminal states omit the future term. Rewards are scaled by 100 for training.
Exploration begins at 100%, decays by 0.992 after each episode, and bottoms out at 5%. Progress toward the goal earns 2 times distance progress. Traps never terminate an episode. Only the finish or the configured step timeout does.
Step: -1; flight activation: -0.5; finish: 100.
Rolling accumulates momentum; braking reduces it to 20%. Flying bypasses low obstacles. Skins are visual only. Brain files preserve online and target weights, topology, epsilon, and update count; imports require matching input and action selections. Replay memory is not transferred. Headless mode suspends the 3D renderer and runs bounded training batches; throughput depends on network size and hardware. Learning is stochastic and does not guarantee convergence.