If it’s overhanging public property it’s fair game. The owner has plenty of fruit on their side too I’ll bet. If they take issue with it they can guide their plant so it’s confined to their property. That being said I wouldn’t be reaching over the fence to yank a cucumber or apple.
If they’ve got 4 fruits left and they’re all hanging over the fence then they just harvested their tree. Let’s not look for hyperspecific edge cases here we’re discussing a rule of thumb.