The phrase "staying by" always gave me pause because it sounds "wrong" to my American English ear. However, it is standard in American Ashkenazi communities (which means most of Jewish America). Just to be clear, it is used instead of "staying with." My understanding is that the Yiddish way of "staying with so-and-so" uses the preposition that translates as "by," so immigrant Jews began saying "saying by" as a literal translation of their native tongue.
As much as I hated the phrase, I've started saying it. You just can't escape it, so you might as well join 'em.