First time you get on a new dirt bike, it is important to adjust it to suit you. Some of the basic stuff is obvious, like being able to reach for the clutch lever, some is not so obvious, like suspension sag.
You should setup the motorcycle properly, even if you don't want to race but just ride around for fun. A properly adjusted bike makes it that much more enjoyable and easy.
Second hand bikes may have been setup completely different by the owner while new bikes sometimes are messed up by the teenagers working to put them together - always do your homework and check it.
The areas to adjust are described in detail in these topics:
Line them up with the forks and make sure they are sufficiently high for a good off-road standing position. You should cut them if you keep banging into trees and fit them with good grips. If you must, add anti-vibration stuff.
Don't forget to check the torque of all the relevant bolts and adjust the height of the forks in the clamp. See all the details in this article: Handlebar Setup
Short, well adjusted levers are very important for complete and comfortable control. Position them horizontally, so you can keep one or two fingers on brake and clutch.
Adjust them up/down and left/right until comfortable. I prefer minimal play, since I ride at low revs most of the time.
Rear brake level with the peg and the shifter a bit higher to get a boot under there. Read it all here: Adjusted levers
Throttle play is very important. Read the owner's manual on adjusting it properly. I prefer to have no play or minimal play (1-2mm) for fine throttle control while controlling the brake lever as well.
Since we're working on the throttle, make sure the cables do not bind. This is part of adjusting the play anyhow: rotate the handlebars from end to end and the engine speed should not change nor should there be any problem with the cables being too tight.
Idle is not really tied to the throttle adjustment, but I'll mention it here. Most bikes have an idle adjustment - please read the manual on how to adjust it. If you find yourself stalling the bike too much, then the idle may be too low. If you find having to slip the clutch a lot at low revs, the idle may be too high.
Good suspension setup is very important. Read more Dirt bike suspension setup.
Well, the first thing about tires is to make sure you have the appropriate tires for that you want to ride most of the time. Knobies for dirt and slicks for the track is basic. In between there is a huge selection of dual purpose and specialized tires. Just pick something suitable or stay with what the bike came with, if appropriate.
See the details on pressure and other tire setups here: Dirt bike tire setup.
Bark busters or hand-guards: if you're riding woods, these are a must. I recommend the ones with a metal frame, attached at both ends, especially for beginners. They will not only save your levers but more importantly, your fingers.
Pegs: there are many after-market pegs available, some are wider and some are adjustable in height.
Tip: save the radiator shrouds. The rad shrouds are normally attached to the rads or the rad guards with a screw. When you run aground or into a bush or a tree, this screw will not give way. What is more likely is that the shroud will rip off. To avoid this, replace that screw with a zip tie. It will hold it in place very well BUT it will give way when you do the unavoidable. Always carry some zip ties with you.
Chain - should be not too loose but not too tight - see the owners manual and check it now and then.
Couldn't say it better so here's a quote from the WEC:
The other mod that many riders find helpful is slower gearing. Standard MX bikes are geared to run at speed on wide open tracks and that gearing in the tighter woods can make the bike a real handful. The slower gearing allows you to run in a higher gear through the tight sections which will increase your lap speed. Most off road riders try to find a gearing combination that allows them to keep the bike in 3rd gear in single track sections.
Other dirt bike setup topics:
More interesting stuff to read or watch, related to dirt bike setup: