Answer by Rabbi Yehuda Shurpin, Chabad.org:
Some have made the claim that this is a relatively new custom, but while it is not necessarily a universal or even a Chabad custom, it is indeed an old Jewish practice that goes back at least to medieval times and possibly earlier.
There are a number of reasons given for this custom, on both a basic level as well as a more esoteric one:
By placing a rock or a pebble on top of the tombstone, we honor the deceased by letting people know that the gravesite has recently been visited. When others notice the rocks, they will see that this is a grave visitors frequent, and they too will take an interest in who is buried there, and perhaps will visit the gravesite themselves.
On a more mystical level, the Talmud tells us that reading the inscription on a gravestone can adversely affect one’s Torah learning.
While the Kabbalists explain that in general this warning applies only to inscriptions that protrude from the tombstone and not words engraved into it, Rabbi Yosef Yuzpa Hahn (1570–1637) cites a tradition that placing a stone on the tombstone also helps to avoid any undesirable consequences that would result from reading a tombstone.
The placement of the stone serves as an invitation of sorts for a spark of the departed to come down and rest upon the tombstone for the duration of the visit.
While placing a rock on a tombstone is an old Jewish custom, placing flowers at a gravesite is not. In life, people may enjoy the beauty of their physical surroundings, but when they die, all of their material possessions and beauty are meaningless and left behind. It is only their accumulated spiritual wealth that remains immortal, just like a rock, which stays forever.
1. Rabbi Yitzchak Borda, Responsa Yitzchak Yeranein, Yoreh De’ah 3:2, contends that this custom stems from the Mishnah (Eduyot 5:6), which says that the practice used to be that if one died, and he was guilty of a sin for which one would normally be stoned, the rabbis would place a very large rock on top of the funeral casket to atone for that sin. However, Responsa Rivevot Ephraim 8:51 vehemently refutes this claim. Others make a claim that this is a new custom, and is really an offshoot of a different ancient Jewish tradition of placing some grass on the tombstone. As we shall explain in subsequent footnotes, this custom goes back at least to the medieval ages, if not to Talmudic times.
2. It seems it is more prevalent within Ashkenazic Jewry. See, for example, Responsa Darkei David, Yoreh De’ah 15.
3. See Rabbi Reuven Margalios, Shaarei Zohar 161a, who points to the following episode in the Midrash as possible evidence of the very early practice of this custom: “Once, a student of Rabbi Shimon bar Yochai forgot all of his learning. Not knowing what to do, he went to the cemetery, where he cried and prayed for mercy. There, he was told in a dream the following words: ‘When you will throw upon me three pebbles, I will come to you.’ Not knowing the meaning, he went to an interpreter of dreams, who replied that the word koli, which means ‘pebbles,’ can also mean ‘voice’; thus, the meaning is that if you review your learning three times, it will come back and stay with you. The student did so, and his previous scholarship was restored” (Kohelet Rabah 10:10). It should, however, be noted that Rabbi Margalios himself expresses a number of reservations at definitively pointing to this as proof that pebbles were commonly placed on graves.
4. Hilchot u-Minhagei Rabbi Shalom of Neustadt (Derashot Maharash) 490, cited in Ba’er Heitev on Shulchan Aruch, Orach Chaim 224:8, and Eliyah Rabbah 224:7; Shaloh, Masechet Rosh Hashanah. See also Taamei ha-Minhagim 1069.
5. Based on this reason, one would place the stone onto the tombstone before leaving the cemetery.
6. Talmud, Horayot 13b.
7. Shaar ha-Mitzvot, Parshat Va’etchanan; Mishnat Chassidim, Yom Gimmel. See also Kitzur Shulchan Aruch 128:13.
8. Yosef Ometz, vol. 2, Seder ha-Limmud le-Atzmo. See Responsa Rivevot Ephraim 8:51, who postulates that the reason placing the stone helps may be based on the ancient custom of Ashkenazic Jews to have the tombstone lying flat. Thus, when one puts a stone or pebble onto the tombstone, one is in essence obscuring part of the text, which serves as a reminder about the consequences of reading an inscription and also prevents a visitor from fully reading the obscured inscription, pre-empting any effects that reading the text in its entirety may have. Based on this, one would place the stone upon the gravestone when he arrives.
9. Preface to Siftei Tzaddikim by Rabbi Solomon Mutzafi. Based on this reason, one would place the stone upon the grave as soon as he arrives, and he would take it off right before he leaves.