Access via private road for building purposes

[Devondumplin]

We are renovating a house that has a boundary with a private road. There is a long wall containing the house itself then a gate and a brick built wall that is adjacent to this private road. The wall is being removed and will be rebuilt in part because of tree damage causing collapse but also to create an extension at the ground level. There is a significant amount of work that needs to be done to the exterior wall because of pointing/ chimneys etc.
We had a face to face conversation with the owner of the private road some time ago explaining that we would need to access the road and put up scaffolding and were given verbal permission
Our builders have begun the work but the owner has been screaming at them that they cannot park or have trucks loading/ unloading and that they will call the police. My contractors have previous history with the Lane owner which ended badly. The owner does not want this person on their land.
My builders are under instruction from me to be considerate, not to park except for delivery/collection. I have arranged other parking for them. Other neighbours have confirmed that they have been considerate, pleasant and have kept the site tidy.
Where do I stand regarding access to my property via this private road? The owner is threatening me with legal action

[efpom]

This might be useful
https://www.legislation.gov.uk/ukpga/1992/23/contents

[Devondumplin]

Thanks for the link