So, where do I start troubleshooting the problem? I had visions of removing and tearing apart my panel, and trying to remember what I did 20 years ago, when I built it. And this is prime float flying season.... But I figured it had to be power, either at the live switch end, or a ground. So I started with the switch, which was easier to get at. I managed to pry apart one side of the panel enough so I could get a multi meter probe in and start poking contacts. First the power side of the switch, and it was live, so that was good. Then turned on the switch and poked the other side, and sure enough it was dead. I was actually very happy then, to find the problem so quickly.
So I removed the switches, and here is what I found. I had three of them of the same type, and the faulty one actually came apart in my hands when I removed it.

Seems that the vibration (and 15 years of usage) over time loosens up the clips on the side that hold the switch together so that eventually the poles no longer make contact in the On position. It also explains another intermittent problem I had been having with my radio since last summer, when it would sometimes cut out when I go to transmit. But it would always come back up on its own, and I would continue on. I could never figure that one out, and thought there was a problem with my mic gain (started happening after I had a headset serviced). But I guess the poor connection on the switch couldn't deliver enough current during transmit (which takes extra power) and the radio would shut down. However, this time it would not come back up, and the transponder went dead as well.
Anyway, went to my local electronics shop and got some new "industrial strength" switches. Was not easy soldering in new switches/wires through a 2 inch wide opening at the edge of my panel. But I got it done, and was out flying again today. And all is good now.
So I thought I would share my experience to those still building, to not scrimp on quality switches for your panel. I can't even remember where I got mine, but can see now, they were definitely not the best.