In our latest research, two Reinforcement Learning (RL) based traffic control solutions applicable in urban traffic network considering future highly automated cars are presented. On the one hand, RL is applied for Variable Speed Limit (VSL) control for urban intersection. Instead of developing yet another adaptive traffic light, a novel approach is introduced where only the speed limits are regulated (dynamically) in the vicinity of the junction. A Multi-Agent Reinforcement Learning (MARL) framework is utilized for setting optimal speed limits to smooth traffic flows and to reduce accident risk. Beside the intersection level control, another solution is proposed using Deep Deterministic Policy Gradient (DDPG) RL for network level, optimal routing of automated vehicles. In this approach objective functions are defined that balance individual preferences with societal needs in traffic flow assignment via multifaceted reward system which includes personal and social efficiency, safety, fuel consumption, as well as driving comfort. The efficiency of both the intersection and network level approaches was tested applying high-fidelity microscopic traffic simulation (SUMO), justifying that RL solutions might have significant implications for the future of urban mobility, paving the way for smarter, safer, and more efficient road networks.