I believe both, sometimes the document alone may be audited to see that it contains all the relevant info and appears suitable & sufficient, but the only real way to check compliance is to visit the premises. How far to dig when inspecting will be down to FRS policies.
As for things going wrong later when they have left building...in my opinion there is no longer a problem as responsibility lies with responsible person not the enforcing authority.
Ios could say that the parts of premises audied at the time of inspection appeared satisfactory..but they may not have looked at all of the premises and there is no legal requirement to do so.
Consider a policeman stopping your vehicle and pointing out a defcetive brake light, you promise to get it fixed and drive off. Around the corner you plough into a bus stop killing ten people because you also have defective steering & brakes. Would Mr Plod be held responsible because he didn't check all safety matters when he stopped you..I think not.