My professor's notes state that the environmental radiation that enters it is almost imposible to get back out of the cavity
That is a little strange formulation, as if somebody would want to get the radiation back out. The idea here is that EM radiation has some properties, such as direction and frequency (distribution of frequencies); and when radiation beam with definite properties is propagating from outside through a small hole into the cavity, overwhelming majority of intensity does not get reflected back through the hole, but it gets absorbed or reflected (or both) to other directions, that imply further absorption or reflection inside the cavity.
Original directions and spectrum of the incoming radiation eventually die out; this happens in steps, during each little reflection or absorption; for spectrum in particular, this happens only via absorptions. So the radiation that comes off the cavity through the same small hole is a mix of many reflections and thermal emissions. This radiation coming off the hole has properties that are nothing like the properties of the previously absorbed non-equilibrium radiation, and do not reveal any of its original properties. When the radiation inside the cavity (except for the part that just came in through the hole) is in a state of thermodynamic equilibrium, the only thing the radiation that came out through the hole reveals, is temperature of the insides.
and that the black body isn't the cavity itself but its actually the hole.
Almost correct, but the hole itself is not enough. The cavity insides have to be effective in equilibrating the injected radiation; a perfectly empty and perfectly reflecting cavity would not work. Either the walls have to be covered with absorbing material, or some body made of such material has to be inside. Only then the radiation inside can evolved towards equilibrium state, and then, from the point of view of an outside observer, the hole is behaving as a one-sided black body surface patch - it absorbs all radiation coming in from one half-space, and does not reflect anything back, and it emits some thermal radiation that is not correlated with the radiation that came in before, but reveals only temperature inside.
Wouldn't that mean that it is equally difficult for the thermal radiation produced in the cavity to exit through the hole?
No. The radiation inside is assumed to be close to equilibrium, so it moves in all directions, so some intensity of it will leak through the hole in all possible directions; this is certain to happen. Again, what was meant is that the hole is so small, and absorption inside is so good, so that almost none of the intensity that went in, goes back with the same spectral features through the hole. The equilibrium radiation inside is assumed to be present and to go in all directions, so some of it goes through the hole and that is all that comes out of the hole.
It's like shouting into an open door to a large room full of people talking loudly; the only thing that comes out the door is the same intense noise as before, you won't hear your echo coming back from the open door. The people in this analogy serve as the absorptive body in the cavity. Or, think of sending water waves from one side of a pool into a stormy water full of people swimming and making their own waves; the waves coming back at you are the same as before and reveal nothing of the waves sent in from the side.
And if it is wouldnt that make that the amount of radiation the black body reflects equal to the amount of thermal radiation it emits ?
This cannot happen, because it would reveal more than temperature of the insides - it would reveal intensity of the radiation that came in.
Reflected radiation always reveals some properties of the incoming radiation, such as direction, or color (spectrum). Perfect black body does not do any such reflection or informing on the past absorbed radiation properties; it radiates its own "thing", as non-informative as possible.
A hole in a reflecting cavity cannot itself reflect, because it is just empty region of space - there are no charged particles there. The only thing that can reflect are the cavity walls inside, but since the hole is small, most of the first reflection does not go through the hole, but reflects again and again inside.