Actually, let me add about the strap. I think its the best bit of the value pack. When playing outside like, I have the strap around my hand, in case I drop it (which I have done - only for the PSP to hang gracefully from my wrist). It's a saviour.
The case is great for keeping the PSP in my pocket. However, It means if you want to carry more games out and about you'd need a differnet case.
So to conclude:
Value Pack:
Strap - very useful (See above)
Case - are you a casual on the move gamer? Travelling to work, school, whatever? Then this is the case for you, since it will fit in a pocket and you can whip out your PSP in a blink of an eye. If you are going to mainly use your PSP on longer journies though, and you have some other kind of bag with you, the case may not be worth it as you cannot carry extra games.
Note: The case doesnt come with any type of seal (ie zip), the PSP may fall out if you are vicious with it. I got a woman to sow a zip on mine, so now it's boss.
Memory Stick - too small. Good for saves and a few MP3s. You will want a bigger stick. I have sold my 32mb MS.
Headphones and Remote - I like these headphones. As Bast says, great quality sound, but this is also the case with other headphones. They seem hard in my ear though. If you've already got headphones, do you need it?