Yes but these things are already problems for human-operated cars. You just need to drive slow and be careful on some roads. You need training, instructions, markers, specialized vehicles, or prior experience on some patches of terrain. These solutions have analogs in the automated driving realm.
I could see autonomous vehicles working in a city, but not out in the country where a driveway might be an unmarked dirt path.