Starting at Bronson Canyon-Griffith Park is a better option if you don't mind a little longer hiking, because you can easily park your car inside the park. The park is public and open from sunrise to sunset. So you will just have to make sure you get back before the gate closes at sunset.
Bronson Canyon-Griffith Park
3200 Canyon Dr
Los Angeles, CA 90068
If you come back using the Hollywood drive on the east side of the area, you can have a really good view of the sign and take beautiful pictures with the sign as the background. However, you will have to go through some restricted trail which gets a bit slippery. It's not recommended to do that path with very little kids. However, my kids enjoyed a lot. We had good view of Griffin Observatory and the other side of the city. You need to hike to the big tank after you see the observatory, and follow the little trail after the tank. At the end of the trail, you will get to the cave. Don't go straight down to the cave as it's too steep. You need to walk around and get to the north west side of the cave and then get down slowly.
The cave is man-made, but worth to explore.