The name Byton means the 'tun of Boi, Boia or Boiga'

6.5km from Wigmore, just to the south of Byton church can be found what is described as a motte with traces of a former shell keep and possible gatehouse to the South. 

The bailey to the north of the motte includes the present church.  At the south end of the bailey is evidence of a small gatehouse with twin towers, like that at Brampton Bryan Castle.

Documentary sources date the castle to c1190.