The standard Windows image viewer is a tool that, although you can get a lot out of it, has serious limitations that won´t be listed here.
If you want to try an alternative, you could opt for OpenSeeIt, a simple, free image viewer for Windows with all the basic requirements, but also with the function that shows you an image and lets you activate the file explorer. Thus you can access the folder structure directly from the application´s interface, browsing your hard drive comfortably without needing to close the programm.