Gemara

Question: A Beraisa teaches that "everyone joins to make a Zimun - Kohanim, Leviyim and Yisraelim." This is obvious!

Answer: The case is, Kohanim ate Terumah or Kodshim (that are forbidden to Zarim), and the Zar ate Chulin;

One might have thought that since the Zar could not eat the Kohanim's food, they do not join. The Beraisa teaches that since the Kohanim could eat the Zar's food, they do join.

Berachos 47a: Rav and Shmuel were eating. Rav Simi bar Chiya came, and ate quickly.

Rav: Do you want to join for a Zimun? We already finished!

Shmuel: If they would bring (for dessert) mushrooms for me or young fowl for you, wouldn't we eat?! (Therefore, he may join for the Zimun.)

Rishonim

Rif (Berachos 35a): We learn from Shmuel that whenever if they would bring something and they (everyone) could eat from it, they join for a Zimun.

Rambam (Hilchos Berachos 5:6): If Kohanim ate Terumah together with Yisre'elim eating Chulin, they are Chayav to make a Zimun, just like they must say Birkas ha'Mazon.

Hagahos Maimoniyos (4): My Rebbi said in the name of the Rashbam that if two ate together in a group that can make a Zimun, and one is careful not to eat bread of Nochrim, and the other eats bread of Nochrim, since the latter can eat with the former (from his bread), they join. If three vowed not to benefit from each other, they do not join. Likewise, if a Kohen and a Zar ate together, and the Kohanim eats Chalah and the Zar eats bread of Nochrim, and the Kohen does not eat bread of Nochrim, they do not join.

Rosh and R. Yonah (Berachos 7:2 and 33a DH u'Ma'aser): The Rashbam says that if three ate together, and one is careful not to eat bread of Nochrim, and the others are not careful, even though the one who is careful cannot eat the others' bread, they join. He proved this from Erchin. Even though the Zar cannot eat Kodshim, since the Kohanim could eat Chulin, they join.

R. Yonah: The Rashbam says that since two of them are careful not to eat bread of Nochrim, and the other eats it, they cannot eat it. Since they are careful, for them it is like Isur. Even so, they join.

Beis Yosef (OC 196 DH Kasav): Also Tosfos, the Rashba, and Mordechai say so.

Rosh and R. Yonah (ibid.): The Rashbam learns from here that if three vowed not to benefit from each other, they do not join. Even though they could permit their vows, now, they did not

R. Yonah: Erchin 4a proves also this. It says that since the Kohanim could eat the Zar's food, they join. This implies that if there was one who could not eat the others' food, and they cannot eat his, they do not join. Also the Ri said so.

Tosfos (Berachos 45a DH Achal): We do not make a Zimun with one who ate Tevel. What is the reason? A Kohen who ate Terumah joins with Yisre'elim, for he could eat with them. Also here, the one who ate Tevel could eat with the others! I answer that eating Tevel is not called eating, for even Kohanim may not eat it, but eating Terumah is proper at least for the Kohen. Likewise, if all three ate Tevel, they do not join. We say that if one stole wheat, made bread, ate it and blesses on it, this only angers Hash-m (Tosefta Sanhedrin 1:3).

Poskim

Shulchan Aruch (OC 196:3): If three ate together, and one is careful not to eat bread of Nochrim, and one is not careful, or one is a Kohen who eats Chalah, even though the one who is careful cannot eat with the other, and a Yisrael cannot eat with a Kohen, since the one who is not careful can eat with the one who is careful, and the Kohen can eat with the Yisrael, they join for a Zimun.

Magen Avraham (1): This is difficult, for in YD (112:15), the Rema permits eating with those who eat bread of Nochrim, to avoid enmity! I answer that we are concerned for enmity only when they eat in one group. Here, each eats from his loaf. The Rema there connotes like this. The Agudah writes that if one ate cheese and two ate meat, they join, since the one who ate cheese can eat with them. He said that Semak (109) agrees. Also the Roke'ach (Shabbos 334) says so. If Tish'ah b'Av is on Shabbos, if the one who ate cheese is careful not to eat meat, they do not join, since none could eat with the others (the Agudah). Hagahos Minhagim and Maharshal challenged this, for they can eat bread together! It is hard to oppose all the Ge'onim (i.e. Poskim) I mentioned above. We can say that they hold that one of them must be able to eat the bread that the others ate. If not, Kohanim and a Zar could eat other bread, like the Beis Yosef said! Here, the one who ate cheese cannot eat the other bread, since it is dirty from meat, like we say in YD 89:4. A Hagahah in Mordechai Katan says that one who ate milk blesses (to exempt the others), for he could eat with them, but not vice-versa. This connotes this is only if he ate milk, but not if he ate hard cheese, for the custom is not to eat meat after cheese. Regarding Tish'ah b'Av on Shabbos, even if he ate milk, they do not join. Hagahos Minhagim connotes that even if there are many, they cannot make one Zimun together, for one cannot exempt another (whom he cannot eat with). Rather, each group makes a Zimun by itself. Who will be lenient against all these Rabanan?! Hagahos Minhagim did not argue with them. He merely asked a question. Every question has an answer. One should not be lenient; people say so. Birkas ha'Zevach asked that they can eat meat after waiting the proper amount of time! This is wrong, for in any case one must say Birkas ha'Mazon before having meat. If one will wait six hours, the food will be digested! Also, there it seems that one must bless Birkas ha'Mazon first. (I.e. if they continued eating bread or Pareve food, the food would not be digested, but due to laws of meat and milk, one must say Birkas ha'Mazon before having meat - PF.)

Kaf ha'Chayim (12): Eliyahu Rabah and Mekom Shmuel agreed with the Magen Avraham. It is better to be passive (and not make a Zimun).

Kaf ha'Chayim (10): The one who avoids bread of Nochrim answers Amen to those who eat and bless on it, just like we make a Zimun with them. Only if something is truly Asur, we do not answer Amen, for the Berachah angers Hash-m.

Shulchan Aruch (ibid.): However, if Kohanim and a Zar ate together, and the Kohanim eat Chalah and are careful not to eat bread of Nochrim, and the Yisrael eats bread of Nochrim, they do not join. Likewise, three who vowed not to benefit from each other do not join.

Beis Yosef (DH veha'Ta'am): The possibility to permit their Nedarim does not help. Kohanim eating Terumah join because they could eat with Yisre'elim. We don't say that Yisre'elim could eat with the Kohanim, if we would permit (Mevatel) the declaration of Terumah! Rather, because it was not yet permitted, they were not able to eat it. The Agur (243) says that some say that even if a Kohen eats Chalah or Terumah and he is careful not to eat bread of Nochrim, and the Yisrael eats bread of Nochrim, they join for a Zimun. If they would have a meal of Chulin bread of Yisrael, they could eat it. This is from Shibolei ha'Leket (151). The Gemara in Erchin refutes this.

Taz (2): I say that if bread of Yisre'elim is available, even if they did not eat from it, they could eat from it, so they join. The Beis Yosef agrees to this. He found difficult the Halachah in Siman 197:1, based on Berachos 47a. The Rif rules like Shmuel, that whenever if they would bring something and they could eat from it, they join. In the case in the Gemara, they were not liable in a Zimun at all. Even so, since if they would bring something else, they would eat, this reasoning enables a Zimun. All the more so, if they all ate (together), just one cannot eat with his friend, surely this reasoning (if they would have bread of Yisrael, all could eat it) enables a Zimun!

Mishnah Berurah (8): They join only if all can eat one bread, and here they cannot.

Mishnah Berurah (9): If two cannot benefit from one due to a vow, they join, for he can eat their bread. If also he may not benefit from them, they do not join. If one ate milk or cheese and the others ate meat, they join, for the first may eat their bread. Even if their bread is dirty with meat, if he cleans and rinses his mouth, it is permitted. The custom is that the one who ate cheese blesses for them, but not vice-versa, for he caused the Zimun. If he ate hard cheese, they do not join, for the custom is not to eat meat afterwards (even) through cleaning and rinsing. The same applies to people careful not to eat meat from Rosh Chodesh Av until Tish'ah b'Av, except for a Seudas Mitzvah. However, if they ate a k'Zayis of bread before the meat eaters began eating meat, they were able to eat together, so they join. The same applies if one of them ate a k'Zayis of bread not dirty with meat or milk. It joins all of them, for all could have eaten from it.

Kaf ha'Chayim (13): If doctors told Reuven not to eat meat, for it harms him, and his bread contains milk, and he eats with people eating meat, all agree that they do not join.

Kaf ha'Chayim (15): Tif'eres Shmuel (on the Rosh, Berachos 7:3) says that there is no source from the Gemara that the one who could eat with the others must bless for them, but it is reasonable, and I received this Halachah.

Kaf ha'Chayim (16): Almost all Poskim say that since they did not permit their vows, they do not join. Gan ha'Melech said that if they make a Zimun, they do not lose. The Birkei Yosef found this astounding.

Rema: This is only if each eats from his loaf. If they eat from the Ba'al ha'Bayis' loaf, they join, for they eat from one loaf.

Taz (3): In YD (221:5), we forbid them to eat from one bowl, lest one eat another's portion, unless the Ba'al ha'Bayis refills the bowl after they ate. We must say that we discuss such a case.