One of the most important issues to consider when creating characters to be used in an NLA is standardization. Most NLAs allow you to transfer animation from one character to another. This is typically done through the use of a naming scheme. Keeping the parts of your characters named in a similar fashion makes swapping animation curves relatively easy. For example, if John's arm skeleton follows this naming scheme:
then, naturally, Marsha's would be similar:
Standardization also extends to motions. If your character's walk cycle starts on the left foot, make sure that the run and skip cycles also start on the same foot. This will make authoring a whole lot easier.
Once you get a character with a standard skeleton and a library of standardized motions, you're ready to go to town. Working with an NLA is less like animating in a traditional sense and more like directing. You get the luxury of viewing the scene as a whole rather than as a complex collection of keyframes, otherwise known as a track. As in a nonlinear video editor, each animation track can be moved, cut, pasted, stretched, and blended with other animation tracks to create and sculpt motion to the animator's content.