fix confusion between "input" and "output"

The code relied on number_of_inputs and number_of_outputs being
the same, which has always been the case so we never noticed. There are
also tests now to try to catch that happening in the future.

Also, there was an accessor function bug which ended up accessing a
matrix of the wrong size, which the new testing caught.

Change-Id: I31791444349fb42ca9105b21336a308277d3c74e
4 files changed