Can you take your picture of the tree and go to the local garden centre? A specialized centre where that's all they do, not the ones that pop up every spring attached to home hardware and grocery stores.
I've done that before with mystery trees, shrubs and flowers. Usually the folks in the centre are a great help, they should be able to identify it for you and maybe offer you options to reduce the berries.
Plus, if they have no way to stop the berries, they'll be able to sell you a non-fruit bearing tree to fill the hole once you get rid of that one
