# MDP quadcopter stabilization

I've designed a quadcopter and have it printed out on a 3D printer. Now I need to control it.

I have formulated an MDP (Markov decision process) and want the helicopter to learn when it is in a stable hovering position.

I have an on-board INS (inertial navigation sensor) that outputs $(x,y,z), (\dot{x}, \dot{y}, \dot{z}), (\phi,\theta,\psi), (\dot{\phi}, \dot{\theta},\dot{\psi})$ - [position], [velocity], [attitude] and [attitude rotation speed] respectively).

Therefore, the MDP is formulated as follows:

States:

• $x$
• $y$
• $z$
• $\dot x$
• $\dot y$
• $\dot z$
• $\phi$
• $\theta$
• $\psi$
• $\dot\phi$
• $\dot\theta$
• $\dot\psi$

Actions:

• rotor1
• rotor2
• rotor3
• rotor4

Rewards:

• 1 if hovering; 0 otherwise

The trouble is, I don't know how to get a transition probability matrix. Do I get this from flight data? How best to go about this?

Is it necessary to build a quadcopter in a physics simulator (I've never done this before...)

Many thanks in advance,

-
There is not enough information to answer this question; however, if possible you should consider your rewards as $+\infty$ for hover and $-\infty$ for not. Depending on the type of learning model you use, the basis functions need not be bounded (ie a polynomial neural network). Letting the outputs be unbounded (when possible) generally leads to better behavior. –  Arkamis Sep 18 '12 at 17:37
I'll make it +\infinity -\infinity... –  Eamorr Sep 18 '12 at 19:39

## 1 Answer

Your question is not answerable. You do not have a specific question on a specific topic. You have a challenging engineering project, and design decisions are not to be made lightly. Moreover, the specifications you provided are, to put it diplomatically, scandalously incomplete!

If you want to play around with RC aircraft and flight control systems, then get an RC airplane that is stable. Helicopters are a nightmare: they are highly unstable and thus hard to control, and the rotating blades can maim or kill you.

If you still want to go ahead, then make sure you obtain a good model of the dynamics of the quadcopter. Then read all the papers you can find on flight control systems for helicopters (take a look at Vijay Kumar's work). Build a SIMULINK model of the quadcopter, and only then can you think of MDP stabilization.

-
I think that the risk of maiming or killing from a tabletop quadrotor is actually fairly low. They're actually kind of cute: youtube.com/watch?feature=player_embedded&v=YQIMGV5vtd4 –  Arkamis Sep 18 '12 at 17:35
@EdGorcenski: The word "quadrotor" does not say anything about the size of the aircraft. It merely specifies the number of rotors. There are controls professors who refuse to use RC helicopters in their labs because of the risks, and not only in the lawsuit-happy U.S.. –  Rod Carvalho Sep 18 '12 at 18:05
It's a small plastic quadcopter about 1/2 meter in diamater. –  Eamorr Sep 18 '12 at 19:38
@RodCarvalho when you say my spec is "scandalously incomplete", could you please let me know what else I need to specify. –  Eamorr Sep 18 '12 at 19:38
@Eamorr: To design a controller, you need to know the dynamics of the aircraft. That involves geometry, mass, moment of inertia, center of mass, thrust power, dynamics of the actuators, what kind of sensors you're using and what their characteristics are, plus what the design specifications are. First, you need to know what you have. Then, you need to know what you want to do with what you have. Have you taken a look at Vijay Kumar's papers? He also gave a TED talk. Your project is fascinating, but not at all trivial. –  Rod Carvalho Sep 18 '12 at 20:09