Do I really need C to program RS08?

There's no reason not to use C to program even the smallest computers. C compilers improve on assemblers by:

• Allowing the developer to work in the problem space (choosing the symbolic operations), not the solution space (choosing the machine instructions).
• Calculating opportunities for optimization with both the algorithm and target hardware in mind.

RS08 has no Negative bit in its condition codes. Can you do signed math operations?

Yes.

RS08 has no stack: how can you perform nested function calls?

C6808 creates a stack through program code: it preserves the Shadow PC (SPC) in a local variable and restores it prior to returning.
As an optimization, it eliminates this code whenever possible. When a function calls no subordinate functions, or when the call to a subordinate function happens at specific points in program flow, the compiler eliminates the saved PC and the memory location it would require.

This approach uses no extra memory compared to a hardware stack.