Farleigh Hungerford Castle was built sometime between 1369 and 1380 by Sir Thomas Hungerford, of Heytesbury – the first Speaker of the House of Commons (c1377). His family and their descendants continued to live here for nearly 300 years and were much connected with Wiltshire. The Hungerfords were one of the richest families in England for many years and owned land from Farleigh to as far as Salisbury. Sir Thomas Hungerford served as Steward of the Household of John of Gaunt and Bailiff for the Bishop of Salisbury. He bought Farleigh Monford House in Somerset in 1369 from Sir Henry de Montford and transformed it into Farleigh Hungerford Castle. He was knighted in 1377 and became Speaker of "The Bad Parliament" through the patronage of his friend John of Gaunt, the First Duke of Lancaster. He was also Sheriff of Wiltshire (1355 - 1360).

At that time, Farleigh Castle comprised of four lofty towers, 60 feet high, and two embattled gatehouses as part of its fortifications with high embattled walls and a moat and a drawbridge thrown across it. It was a strong fortress, set on a steep hill, with a stream at the rear. But Sir Thomas had neglected to procure the Royal Licence to execute these works and was obliged to pay a fine of 1,000 Marks to King Richard II as a conditional pardon. He was married to Joan Hussey, the daughter and co-heiress of Sir Edmund Hussey of Holbrook and died on the 3rd December 1397. Sir Thomas was buried in the Chapel of St. Anne, the parish chapel outside the castle. His wife, Joan, died on the 1st March 1412 and was buried beside him. The tomb of Sir Thomas and his wife Lady Joan Hungerford (1397 and 1412) lie within the Old Chapel of St. Leonard, surrounded by iron railings.

Sir Thomas was succeeded by his son, Sir Walter Hungerford, who was known as the First Lord Hungerford. He was Knight of the Garter and Lord High Treasurer of England (1426 - 1432). His principal residence was Heytesbury and he was married to Catherine Peverill, the daughter and heiress of Thomas Peverill. He was also First Lord Sheriff of Somerset and Dorset and Sheriff of Wiltshire (1404 - 1407). Like his father, he was also Speaker of the House of Commons and in 1426 was admitted to the House of Lords as Baron Hungerford. He distinguished himself in the French wars of Henry V, especially at Calais, and was rewarded for his services by a grant of 100 Marks per annum, payable out of the town and Castle of Marlborough. He fought at Agincourt and at the Siege of Rouen and was the Executor of Henry V's will and a Member of Council under Henry VI. In his time at Farleigh he greatly enlarged the outer court of Farleigh Castle, enclosing the parish church. When he died on the 9th August 1449 he was succeeded by his son, Robert.

This Robert Hungerford was known as 3rd Lord Hungerford and Lord Moleyns by right of his wife. He was married to Eleanor Moleyns, the daughter of William Lord Moleyns. While fighting abroad he was captured at Castillon in 1453 – the very last battle of the Hundred Years' War – and imprisoned for seven years in France. When he returned to England, he joined the Lancastrian army in the Wars of the Roses (1455 - 1485), but was again taken prisoner at the Battle of Hexham in 1464 and was attained. As a result, by a Bill of Attainer, King Edward IV declared Robert Hungerford guilty of treason, and without the inconvenience of any trial, his estates were forfeited to the Crown and he was beheaded at Newcastle in 1464 for his support of the Lancastrian cause. They had two sons, Thomas and Walter. Both Baron Hungerford and his wife, Eleanor, are said to be laid to rest in Salisbury Cathedral.

Their eldest son, Sir Thomas Hungerford, supported Edward IV but afterwards falling off, and endeavouring the restoration of King Henry VI, he was seized and tried for his life at Salisbury, where he was judged a traitor and beheaded in 1469. This resulted in Farleigh becoming crown property for more than 20 years. While in Crown hands, King Edward IV gifted Farleigh Castle to his brother, Richard, the Duke of Gloucester – later King Richard III, who in turn, granted it to John Howard, Duke of Norfolk. However when King Richard was defeated by Henry VII at the Battle of Bosworth in 1485, John Howard lost his life at the same battle.

Fortunes were turned however for the Hungerford family when Sir Walter Hungerford of Heytesbury and Farley, a nephew of the above mentioned Sir Robert Hungerford, the last proprietor of Farleigh, was knighted on the same battlefield by Henry VII and a year later he recovered Farleigh Castle in 1486 when the King gave it back to the Hungerford family. He had been an Esquire of the Body of Edward IV and had served under him in France in 1475, appointed Lieutenant of Dover Castle in 1472 and Sheriff of Wiltshire in 1510 and served as a Privy Councillor. He was married to Jane Bulstrode, the daughter of William Bulstrode and died in 1516. He was buried in Farleigh Castle and was succeeded by his son, Sir Edward.

This Sir Edward Hungerford was Sheriff of Wiltshire in 1517 and was married three times – his first wife was Jane Zouch. When he died, just six years after his father in 1522, he left his estate in his will to his widow, Agnes Cotell, but she was arrested and hanged at Tyburn in 1523 on a charge of arranging the murder of her previous husband, John Cotell, after she strangled him and burnt his remains in the kitchen furnace at Farleigh Castle in order to free herself so she could marry Sir Edward and get her hands on all his wealth. During his time at Farleigh, Sir Edward made additions to the chapel and castle.

After Agnes’ execution, Farleigh Hungerford fell into the hands of Sir Edward’s heir, the twenty one year old Sir Walter Hungerford, the Third Lord Hungerford, who was one of Henry VIII’s squires at the time. Like his father, Sir Walter was married three times. His first wife was Susan Danvers, the sister of Lucy Danvers who was married to Sir Henry Bayntun of Bromham House, Wiltshire and the daughter of Sir John Danvers, of Dauntsey, Wiltshire. After the death of his first wife he married Alice Sandes, the daughter of William, Lord Sandes in 1527 but she was accused of poisoning her husband some years later and hanged, along with one of her servants. In 1532 Sir Edward remarried, for the third time, to Elizabeth Hussey, the daughter of John Lord Hussey and in the same year he became very friendly with Thomas Cromwell when the latter made him Sheriff of Wiltshire that year. He is said to have abused his last wife Elizabeth terribly and he imprisoned her for more than three years in the south-west tower of the castle, with little to eat or drink, allowing her to see no one but his chaplain. This tower is still standing and since then has been known as 'The Lady Tower'.

Upon her release from imprisonment in 1536, Elizabeth was driven to appeal to Thomas Cromwell for protection, fearing she might be locked up again and claimed he had tried to poison her on a number of occasions. Cromwell however ignored her plea, seeking to protect her husband who was his friend. In fact Cromwell suggested he be rewarded, approving of his worth, loyalty and efficiency and ensured that he was richly rewarded and also promoted within the peerage to Lord Hungerford of Heytesbury, with the right to sit in the House of Lords and a 'Squire of the Body' to Henry VIII. She was eventually freed in 1536 and married Catherine Parr’s first cousin, Sir Robert Throckmorton and had five children with him.

This Sir Walter Hungerford was Sheriff of Wiltshire (1557, 1572, 1581 and 1587) and was known as the Knight of Farleigh, who recovered and made further updates to the castle while he resided there. He was married to Anne Bassett but after her death he remarried Anne Dormer, the daughter of Sir William Dormer but later accused her of adultry and attempted poisoning her, however the charges were dismissed. But rather than pay the legal fees, he went to prison instead, however he died in 1595 without an heir – his only son, Edward, had pre-deceased him in 1585. The tomb of Sir Walter Lord Hungerford is located in the south east corner of the large chapel.

Sir Walter was the 7th in lineal descent from the first owner, Sir Thomas Lord Hungerford, believed that after his death Farleigh would pass to his mistress, but instead it was settled upon his wife – Lady Anne Dormer and when she died, in 1603, the castle passed to Edward Hungerford – Walter's half-brother, who also made changes to the castle. But this Edward Hungerford too died without an heir in 1607, leaving Farleigh Castle to his widow, Jane, for life with the remainder to his great-nephew Sir Edward Hungerford. This Sir Edward (died 1607) is also buried in the north east corner of the smaller chapel.

This Sir Edward was Sheriff of Wiltshire in 1631 and during the Civil War he was a Parliamentary Commander of the local forces of Wiltshire for Parliament in 1642-45. In October 1642, his neighbour, Sir Edward Bayntun was appointed Commander-in-Chief for Wiltshire County and he too commanded a force for Parliament in the early part of the Civil War and was also a Commissioner of the English Parliament to the Scots Army. Hungerford was very jealous of Bayntun's appointment and the two fellow Commanders are said to have quarreled a lot, each accusing the other of treachery. This eventually led to the withdrawal of Bayntun from active soldiering. Bayntun subsequently made approaches to the King (for which he was imprisoned for some time in the Tower) and the Bayntun family mansion – Bromham House – was taken and burned to the ground by Royalist Forces from Devizes on 5th May 1645, led by none other than Sir Edward Hungerford's half brother, Sir John Hungerford.

Farleigh Castle too was captured by royalist troops, led by a certain Colonel Sir John Hungerford, who was a half brother of Sir Edwards – it wasn’t unusual at this time for siblings to fight on opposing sides. Colonel Hungerford held the castle for two years (1644-1645) before the Parliamentarians managed to capture it back and it was handed back to Edward again, which meant that it was firmly in the hands of Parliamentarian forces and escaped the deliberate and very sad destruction of similar castles, and great houses such as Bromham House, in the area that were destroyed at this time. Sir Edward Hungerford died in 1648 and his wife commissioned, at the time, an elaborate, sculptured, expensive tomb, where he, and she at a later date, were laid to rest which still stands today in the chapel at Farleigh. This fine monument is of black and white marble. The upper slab is in a single piece, 8 feet long by 5 feet wide.

This time Farleigh was inherited by another half-brother, Anthony Hungerford, another Royalist, who died at Farleigh in 1657.

He was survived by his eldest son, Sir Edward Hungerford, MP for Chippenham – an aristocrat, blessed with abundances of wealth, thanks to his inheritance and shrewdness to have been married to an extremely rich heiress, during the restoration of Charles II to the throne. He was known at the time as 'The Spendthrift' and was one of the least worthy members of the Court of Charles II, and spent most of his time entertaining extravagantly and gambled away in succession 28 manors and was eventually forced to sell most of the Hungerford lands, under the necessity of assigning his estate to trustees for the benefit of his creditors. He is said to have wasted £80,000 capital and £14,000 a year. He is also said to have paid £500 for a wig he fancied and lost £50,000 across the green cloth. His passion for gambling as well as the usual wine, women and song, landing himself in debt to the tune of millions in the process, which necessitated the selling off of most of the family lands.

In 1684 Henry Bayntun of Spye Park – the grandson of the above mentioned Sir Edward Bayntun – purchased Hinton Priory from Sir Edward Hungerford and the Manor of Tellisford, Wick Farm, the Manor of Norton St. Philips, the Manor of Wellow, the Manor of Rode, the Manor of Langham and lands elsewhere in the neighbourhood. In fact when any of the Hungerford properties came under the hammer, they were bought up by Henry Bayntun. This was known as "The Great Sale" and in 1686 Bayntun bought the Manor and Castle of Farleigh for £56,000 from Sir Edward Hungerford. This certainly would not have pleased any of Hungerford's immediate family, to see their stately castle pass into outside hands for the third time.

Above this archway there was a single apartment which was a guard-room leading onto the walls. The window of which is still visible today. From this entrance the south and west sides were protected by a narrow moat and on the east and north sides the secluded hill on which it stands defended it as the ground fell rapidly towards the north.

On the 5th November 1842, the ivy growing over the top of the tower caught fire as a result of some careless children and was completely destroyed.
The tower being deprived of its girders, which soon fell down, to reveal what is looks like today.

In 1687, Sir Edward Hungerford also sold the Manor of Ilford to Bayntun and Henry devised it on trust for sale and in 1700 his trustees sold the Manor of Ilford and the Manor of Rowley to William Chanler of Bradford. It is said that Hungerford lived the last 30 years of his life on charity and died in 1711 at a poverty-stricken old age at his only remaining manor – Black Bourton in Oxfordshire.

Henry Bayntun died suddenly in 1691 at the age of 27 but he and his wife, Anne, had resided at Farleigh Castle for a short while prior to this. Henry had been previously living at the family mansion at Spye Park, near Bromham but must have liked Farleigh Castle very much to have left the family home to reside there. In the years 1700-1702, Henry's widow – Lady Anne Wilmot – was forced to sell most of the Hungerford estates, including the manoral lands at Farleigh and the castle to Joseph Houlton. The Houlton family are recorded to have pulled out the many beautiful fireplaces, marble floors and stone door surrounds and transferred all its panelling and carved beams to their other house at Trowbridge, but the bulk of the stones went to build the handsome Farleigh House in the park outside the castle. Sadly from this time onwards, the castle structure went into decay and a ruin by the beginning of the 18th century.

In the early years of its occupation, Farleigh Castle is described as almost unchanged from the time of the old English Barons, but by 1701 a long process of decay had begun to take its toll. The shell of the gatehouse and what was the southern entrance to the castle is still standing and the extent of the enclosed area and the walls and forms of the towers may still be traced. The castle chapel contains many family monuments and paintings. In the chapel's crypt, the coffins of many hungerfords are still visible, several with attached death masks. Today what remains of this once magnificent castle is in the care of the English Heritage

A north view of Farleigh Castle in 1733 from an engraving by Samuel and Nathaniel Buck. The beginning of its decay is evident in the two towers on the right of the picture.