It would be an incredibly narrow, short sighted, snobbish, and quite simply a dumb action to take, in limiting books to those marketed as Literary. The works of Poe, Shakespeare, Borges, Wolfe, Bradbury, Ellison, and legions of others are no less worthy of consideration just because they are authors who work primarily as fantasists.
The Thumbs crew seem to be too well rounded and read to fall into such a limited criteria.
Anyway, some none Pop-Lit recommendations:
The Fifth Head of Cerberus by Gene Wolfe
Shriek:An Afterward by Jeff Vandermeer
Growing up Dead in Texas by Stephen Graham Jones
Non-Fiction:
The Demon Haunted World:Science as a Candle in the Dark by Carl Sagan
Discarded Science by John Grant