Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've designed a quadcopter and have it printed out on a 3D printer. Now I need to control it.

I have formulated an MDP (Markov decision process) and want the helicopter to learn when it is in a stable hovering position.

I have an on-board INS (inertial navigation sensor) that outputs $(x,y,z), (\dot{x}, \dot{y}, \dot{z}), (\phi,\theta,\psi), (\dot{\phi}, \dot{\theta},\dot{\psi})$ - [position], [velocity], [attitude] and [attitude rotation speed] respectively).

Therefore, the MDP is formulated as follows:


  • $x$
  • $y$
  • $z$
  • $\dot x$
  • $\dot y$
  • $\dot z$
  • $\phi$
  • $\theta$
  • $\psi$
  • $\dot\phi$
  • $\dot\theta$
  • $\dot\psi$


  • rotor1
  • rotor2
  • rotor3
  • rotor4


  • 1 if hovering; 0 otherwise

The trouble is, I don't know how to get a transition probability matrix. Do I get this from flight data? How best to go about this?

Is it necessary to build a quadcopter in a physics simulator (I've never done this before...)

Many thanks in advance,

share|cite|improve this question
There is not enough information to answer this question; however, if possible you should consider your rewards as $+\infty$ for hover and $-\infty$ for not. Depending on the type of learning model you use, the basis functions need not be bounded (ie a polynomial neural network). Letting the outputs be unbounded (when possible) generally leads to better behavior. – Emily Sep 18 '12 at 17:37
I'll make it +\infinity -\infinity... – Eamorr Sep 18 '12 at 19:39

Your question is not answerable. You do not have a specific question on a specific topic. You have a challenging engineering project, and design decisions are not to be made lightly. Moreover, the specifications you provided are, to put it diplomatically, scandalously incomplete!

If you want to play around with RC aircraft and flight control systems, then get an RC airplane that is stable. Helicopters are a nightmare: they are highly unstable and thus hard to control, and the rotating blades can maim or kill you.

If you still want to go ahead, then make sure you obtain a good model of the dynamics of the quadcopter. Then read all the papers you can find on flight control systems for helicopters (take a look at Vijay Kumar's work). Build a SIMULINK model of the quadcopter, and only then can you think of MDP stabilization.

share|cite|improve this answer
I think that the risk of maiming or killing from a tabletop quadrotor is actually fairly low. They're actually kind of cute: – Emily Sep 18 '12 at 17:35
@EdGorcenski: The word "quadrotor" does not say anything about the size of the aircraft. It merely specifies the number of rotors. There are controls professors who refuse to use RC helicopters in their labs because of the risks, and not only in the lawsuit-happy U.S.. – Rod Carvalho Sep 18 '12 at 18:05
It's a small plastic quadcopter about 1/2 meter in diamater. – Eamorr Sep 18 '12 at 19:38
@RodCarvalho when you say my spec is "scandalously incomplete", could you please let me know what else I need to specify. – Eamorr Sep 18 '12 at 19:38
@Eamorr: To design a controller, you need to know the dynamics of the aircraft. That involves geometry, mass, moment of inertia, center of mass, thrust power, dynamics of the actuators, what kind of sensors you're using and what their characteristics are, plus what the design specifications are. First, you need to know what you have. Then, you need to know what you want to do with what you have. Have you taken a look at Vijay Kumar's papers? He also gave a TED talk. Your project is fascinating, but not at all trivial. – Rod Carvalho Sep 18 '12 at 20:09

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.