The first stamp to be issued, in England on 6 May 1840, was the Penny Black. It's called the Penny Black because it cost a penny, and it was black. The face on the stamp is Queen Victoria, who was Queen at that time. Just because a stamp is old doesn't necessarily make it valuable. The Penny Black is not rare - 68 million of them were printed - but if you had one in excellent condition it could be worth $1000.
In Australia's early days, stamps were issued by individual colonies. The first stamp issued in Australia came from New South Wales in 1850. The one penny stamp which showed the seal of the colony is worth around $5000 in mint condition. The Kangaroo and Map series, first issued in 1913, were the first real Australian stamps. A whole range of stamps bearing this design was issued, valued from half a penny to £2 (about $4 in today's money). A mint copy of the £2 Kangaroo and Map could be worth as much as $4000 today.