You can go portable or permanent. You can buy about any kind of mount to attach one to your yak. Portables carry their own battery. There are internal and external battery mounts. If portable you just stick the transducer to your kayak. Otherwise most mount in the a scupper hole well.
i power mine with a gell filled battery, one that goes to the kids little cars they ride, got it from wal-mart 25-40 $$ depends on which one. full charge it will last me 3-4 days. have a malibu stealth, put mine in back hatch wires run inside to where i have my lowrance mounted.
I just got one of the Hummingbird PirhanaMAX portable units. All in one including the battery in a pack with a hard bottom and D rings that you can set on the deck, clip in place, suction cup transducer and you are in business. The low end one I got wasn't much over $100 including a battery and charger. Biggest downside is transducer hanging below and a wire running to it. I get in the habit of ignoring branches I can squeeze by or over in the kayak and have to remember i have something hanging off. You can get some duct putty and mount the transducer inside the hull instead, but you have to have a spot for it and it won't be as portable.
I have a lowrance X-4. SImple black and white. I mount mine inside my yak using duct puddy. I power mine on 8 AA batteries (rechargable) that i have turned into a 12v via radio shack. I protect them inside a small waterproof plano box. I get a day or two from the charge but recharging them is easy. Total cost just about $100.