Smart building applications (apps) are faced with the real challenge of unfettered access to mission-critical building resources that makes buildings vulnerable to attacks and occupants to privacy invasions. Existing methods that group users for access control are too coarse-grained to avoid granting over-privileges to apps. Furthermore, they lack means to model, express, and use access patterns that can be critical in securing automated building operations. In this paper, We identify Who, What, and When as the key information dimensions for building apps access control after thoroughly reviewing 125 smart building app publications in two major venues. Our analysis reveals that dynamic access control requires unique access patterns of individual apps, as well as the building and user context. We also observe that existing Building Operating Systems and IoT platforms fall short of sufficiently representing all the necessary patterns, and further discuss future directions for the design of access control systems needed to support building apps.