Hi Nelson,
It is good that the ground on the sockets has no connection to the neutral wire of the mains in your home (if that is what you meant).
Hopefully, there is no any other household applience in your home either, which has a poor or damaged isolation between its ground and the neutral connections which may willy-nilly 'provide' an unwanted direct connection or only a a few kOhm 'leakage' resistance or impedance between the neutral and the ground at the sockets.
Regarding the strange behaviour of the scope's ground probe i.e. when you "
invert the probe position" at the AC input legs of the diode bridge and the effect stops, I think the following:
In an oscillator circuit the components are connected to either a low or a high impedance circuit point with respect to the DC supply rails. Also the components can have either a low or a high oscillating voltage across them, also with respect to the supply rails but of course this can be true directly across a particular component.
So when the ground probe of the scope is connected to the 'high' voltage point of L3 (which is also one of the AC diode bridge inputs), the available voltage difference to drive current into the 'ground' connection will be higher (to better initiate the effect) than in the case the ground croco is tied to the other end of L3. You may say that L3 is totally ground independent because it is inductively coupled to L2 but L2 is already directly connected into the circuit and the leakage inductance and stray capacitance between them may transfer the low and high voltage differences of L2's legs to L3 legs with respect to the ground rails. (The voltage difference across L2 is transformed normally, of course, L2/L3 being a 1:1 transformer.) This is how I think this can happen.
You also posted this:
"I would like to add that if the probe is placed at the exit of the diode bridge rectifier, where it is supposed to be DC, the same behavior occurs,"
Well, if my approach to explain this behaviour when the probe ground was at the input of the diode bridge is correct (what I think it is), then it should be valid for the output side of the diode bridge because the diodes conduct alternately of course, connecting L3 legs to the DC output.
Note C7 may have a role in this too, for it directly shunts one of the bridge diodes, albeit a 4 nF has about 2 kOhm capacitive reactance at 19 kHz, so may not be too significant (but it is present and 2 kOhm is not high to limit current when the peak to peak AC voltage is around 300 V).
The use of an inverter to supply ground independently your measuring instruments sounds good.
Another note: For your own safety, if your device under test is run from the mains and you need to tinker on it, the use of an inverter or an 1:1 isolating mains transformer is also a good idea (the latter is perhaps the best).
Regards and have cooler weather.
Gyula