Possible Causes

I suspect the source to be something to do with the water system. Either mechanical equipment, water hammer or cavitation. The water pipe coming into the house has the highest level of vibrations. Here is summary of the possible sources:

Gas pipeline transmission systems: Initially, I ruled this out as a potential source because the distance from high pressure gas pipelines. However, my experience is seems to be similar to many other people's throughout the world with some documentation suggesting that this is the source (see resources) 

     

Domestic water supply system: This system seem to be the most likely source of the vibration pollution. I have measured the vibrations all over my current house. The levels are pretty consistent throughout the structure and the garage. Both the gas and water lines coming out of the ground seem to have higher vibrations than the surrounding ground. But the water pipe has considerably higher vibration levels. I suspect the this to be the source. We have really high water pressure. All three places we live with this problem have been at the bottom of a hill, maybe this is why the pressure is so high. It seems likely that the source is pumping equipment. 

Well or reservoir : There are some wells in the area and of course the Sunset Reservoir. I don't know much about how these systems operate, but potentially they have something to do with the problem.

 

Street and Building Damage: I've noticed cracks on my street and sidewalk. Here are some pictures. Once I saw the cracks, I see it all over the neighborhood.

And Ulloa St has similar damage to recently poured concrete. Indicated excessive strain.

After some consideration and a few years of red herrings, here are some of  the sources that I have ruled out:

Appliances or mechanical equipment in neighboring buildings: Given that we have experienced the vibrations in three different homes and the pattern and sensations is almost always the same, this seems like an unlikely source. Additionally, there doesn't seem to be any correlation with weather (HVAC equipment). It can occur continuously for hours or only minutes make other types of appliance (refrigerator, washer/dryer) implausible.

Traffic, Muni street cars, or natural causes: This doesn't really fit what we have experienced. The vibrations start abruptly and can last for a long time. Traffic and muni doesn't  makes sense. would probably pass pretty quickly. Also the frequencies seem much to high to be some natural cause like wind swaying the structure or someone doing jumping jacks next door. 

Natural Gas distribution system: While it seems possible, from what I have read, the pressure in natural gas distribution is pretty low. The pressure only needs to be slightly positive to move the gas through the pipes. It doesn't seem likely that the low pressure gas lines would carry vibrations from some remote compressor or utility equipment.

Auxiliary Water Supply System (AWSS): I only recently learned about this system. There is a high pressure water system in parts of San Francisco used for fighting fires. This system is engineered  to maintain high pressure in the event of a major earthquake. This system is gravity fed from a series of reservoirs and tanks. The base operating pressure at lower levels in the system is 140 psi, and can be stepped up to 214 or 328 psi, as needed. This system doesn't have pumps. It seems possible that the high pressure transmits vibrations from an adjacent domestic water treatment or pumping station. Or maybe the vibrations are the result of some sort of cavitation or water hammer. It may be a coincidence, but all three places we have lived have been within a few blocks of this system.