this is pretty straight forward, how big are the batteries? Lead acid batteries dont get much smaller than C-cell batteries. Coin cells don't get much larger than a quarter. There are also standard sizes, such as AA and 9V which may be desirable.

This is a performance issue: higher quality (and more expensive) batteries will have a higher power density. If weight is an important part of your project, you will want to go with a lighter, high-density battery. Often this is expressed in Watts-hours per Kilogram.

Price is pretty much inversely proportional to power-density and proportional to power capacity. The more power you want in a smaller, lighter package the more you will have to pay.

The voltage of a battery cell is determined by the chemistry used inside. For example, all Alkaline cells are 1.5V, all lead-acid's are 2V, and lithiums are 3V. Batteries can be made of multiple cells, so for example, you'll rarely see a 2V lead-acid battery. Usually they are connected together inside to make a 6V, 12V or 24V battery. Likewise, most electronics use multiple alkalines to generate the voltage they need to run.
Don't forget that voltage is a 'nominal' measurement, a "1.5V" AA battery actually starts out at 1.6V and then quickly drops down to 1.5 and then slowly drifts down to 1.0V at which point the battery is considered 'dead'.

Some batteries are rechargable, usually they can be recharged 100's of times.

Since this is a particularly confusing part of measuring batteries, I'm going to discuss it more in detail.

Power capacity is how much energy is stored in the battery. This power is often expressed in Watt-hours (the symbol Wh). A Watt-hour is the voltage (V) that the battery provides multiplied by how much current (Amps) the battery can provide for some amount of time (generally in hours). Voltage * Amps * hours = Wh. Since voltage is pretty much fixed for a battery type due to its internal chemistry (alkaline, lithium, lead acid, etc), often only the Amps*hour measurement is printed on the side, expressed in Ah or mAh (1000mAh = 1Ah). To get Wh, multiply the Ah by the nominal voltage. For example, lets say we have a 3V nominal battery with 1Amp-hour capacity, therefore it has 3 Wh of capacity. 1 Ah means that in theory we can draw 1 Amp of current for one hour, or 0.1A for 10 hours, or 0.01A (also known as 10 mA) for 100 hours.

However, the amount of current we can really draw (the power capability) from a battery is often limited. For example, a coin cell that is rated for 1 Ah can't actually provide 1 Amp of current for an hour, in fact it cant even provide 0.1 Amp without overextending itself. Its like saying a human has the capability to travel up to 30 miles: of course running 30 miles is a lot different than walking! Likewise, a 1Ah coin cell has no problem providing a 1mA for 1000 hours but if you try to draw 100mA from it, it'll last a lot less than 10 hours.

The way the power capability is measured is in C's. A C is the Amp-hour capacity divided by 1 hour. So the C of a 2Ah battery is 2A. The amount of current a battery 'likes' to have drawn from it is measured in C. The higher the C the more current you can draw from the battery without exhausting it prematurely. Lead acid batteries can have very high C values (10C or higher) , and lithium coin cells have very low ones (0.01C)

These are more popular rechargables, they also come in 'standard' sizes. These are a good replacement for standard alkaline batteries in many cases. The battery cel voltage is 1.25V per cell, that's less than the 1.5V of alkalines but more than the 1.2V of NiCads.They like to be charged at about 0.1C but can be discharged at 0.2C

Pros: Good alternative to Alkalines in most situations, high power density, "standard" size, better capability than alkalines, pretty easy to recharge but not as rugged.
Cons: More expensive than Ni-Cads, service life isnt as long, don't like to be used in a high-drain system, self-discharges quickly.

Prices: AA size battery costs around $2 and has up to 2500mAh of charge capacity
Power Density: 100 Wh/kg

These are the latest in rechargable battery technology, and are quickly becoming the most common batteries for consumer electronics like camcorders, cell phones, laptops etc. They are very lightweight, don't mind high discharge rates, and have very high power density. However, they are very delicate and require special circuitry to keep them from exploding! This means that raw LiIon cells are very rare and very dangerous. Most li-ion batteries come with protection circuitry that keeps the battery operating safely. If you want to use LiIons, your best bet is to use camcorder or cell phone batteries and use the charger that matches up with it. LiIon cels are around 3.6V so 3.6V and 7.2V are the most common battery voltages you'll see. They can easily provide up to 1C of current, some can go up to 10C!

Pros: Ultra-light, high power, high capability, high cell voltage.
Cons: Expensive, delicate, can explode if misused!

Prices: replacement "cell phone" batteries cost around $10 and have ~750mAh charge capacity.
Power Density: 126 Wh/kg for lithium ion, 185 Wh/kg for lithium polymer

Lead Acid batteries are the workhorse batteries of industry. They are incredibly cheap, rechargeable, and easily available. Lead acid batteries are used in machinery, UPS's (uninterruptable power supply), robotics, and other systems where a lot of power is needed and weight is not as important. Lead acid batteries come in 2V cells, that means you can have a battery with an even number of volts. The most common voltages are 2V, 6V, 12V and 24V.

Pros: cheap, powerful, easily rechargeable, high power output capability
Cons: very heavy, batteries tend to be very large bricks because power density is very low

Prices: A 12V lead acid battery with 7Ah of charge should run about $20
Power Density: 7 Wh/kg.

Alkaline batteries are the most common batteries you will come across. They are the ones sold in every store, so they're great for projects that need to be 'user serviceable.' They have higher power density than NiCads and slightly better power density than NiMH. However, they are one-time use. Cels are 1.5V, and availble in sizes from coin cells to AAAA to D cell. One nice thing about having multiple sizes with a standard voltage is that you can always just specify the next size up when you need more capacity and capability. You'll get the published capacity rate if they are discharged at about 0.1C.

6V lantern batteries are very large alkalines made of a couple large cells, they're rather convenient in that they're available in many stores, have massive capacity and capability and you can clip/solder onto their tabs pretty easily.

9V batteries are a strange case: they're actually made of 6 very small 1.5V batteries, pretty much the size of coin cells. As a result they have very low capacity and capability and are very expensive. If you are drawing more than 20mA then they are probably not a good idea to use

Prices: AA size battery costs about $1 and has up to 3000 mAh of charge capacity
Power density: 100 Wh/kg

Pros: Popular, well known, safe, long shelf life
Cons: Non-rechargeable, low-capability

These are the older rechargables that were popular for a long time. They come in 'standard small' battery sizes like AA and C. They are not used as much these days because NiMH batteries have much higher power density. However they are cheaper, and are still used in many cordless phones, solar lights and RC cars where performance is not as important as price. Another nice thing is they discharge slower than NiMHs. (That is, left alone, they will retain their charge longer) Battery cells are 1.2V, often bundled in "packs" of 3 to make 3.6V.

Pros: Inexpensive, rugged, come in "standard" sizes, easy to recharge
Cons: Lower power density, requires "full discharge/recharge" cycles every once in a while to reduce 'memory effect' (the growth of crystals on the battery plates), contains toxic metal

Prices: AA size battery costs around $1 and has up to 1000mAh of charge capacity
Power Density: 60 Wh/kg

Most of the lithium batteries you'll see are in coin/button cell form.Coin cells are small discs, often Lithium cells are used (3V) but Alkaline, zinc air, and manganese are also used (1.5V).

They are very small and very light, great for small, low-power devices. They're also fairly safe, have a long shelf life and fairly inexpensive per unit. However, they are not rechargeable and have high internal resistance (which is what makes them safe) so they can't provide a lot of continuous current: 0.005C is about as high as you can go before the capacity is seriously degraded. However, they can provide higher current as long as its 'pulsed' (usually about 10% rate)
One of the most popular coin cells in use right now is the CR2032 which is 20mm diameter x 3.2mm thick, provides 220mAh at 3V. Lithium coin cells can get as large as the CR2477 (24mm x 8mm) with a capacity of 1000mAh for $3.50

The only other lithium cell you'll see around is the CR123, which is a 3V cell thats a bit thicker than a AA battery and a bit shorter too.

Pros: Light, high-density, small, inexpensive, high cell voltage, easy to stack for higher voltages, long shelf-life
Cons: Non-reusable, low current draw capability, needs a special holder

Prices: CR2032 are around $0.35 (220mAh) CR123's are $1.50 (1300Ah)
Power density: 270 Wh/kg