Local 87, the union representing 6,000 janitors in the private sector, said federal agents went to multiple downtown office buildings late Friday afternoon, but that building security kept them from entering.