I will argue that transcription ‘factories’ are central organizers of the human genome during interphase, and that proximity to an appropriate factory determines the activity of a gene. The nucleolus is the prototypic factory; it is a place where many rRNA genes are efficiently co-transcribed by local concentrations of RNA polymerase I. Analogous clusters of RNA polymerase II in nucleoplasmic factories make protein-coding transcripts. I begin by describing new forces able to drive genome organization uncovered using Brownian dynamic simulations. For example, a diffusion-based “osmotic ratchet” can force bound “slip-links” like cohesin to “convergent” CTCF sites without the need for any motor activity. Additional simulations point to an unforeseen ‘bridging-induced attraction’ that can assemble factories and organize interphase chromosomes at all scales. This organization leads naturally to an explanation of how gene activity is regulated: a promoter is only likely to initiate if tethered near a factory containing appropriate factors. As motifs like enhancers, silencers, insulators, barriers, and boundaries are transcription units, they would work by tethering target promoters close to, or distant from, suitable factories; although we might name the motifs differently, they are all just transcription units influencing promoter-factory distance (and so initiation frequency).