Antennas are required now to be compact and mobile. Traditional horizontally polarized antennas are placed in a quarter wave distance from a ground plane making the antenna system quite bulky. High impedance surfaces are proposed for an antenna ground in close proximity. A new method to achieve a high impedance surface is suggested using a metamaterial comprising an infinite periodic array of conducting loops each of which is loaded with a non-Foster element. The non-Foster element cancels the loop's inductance resulting in a material with high effective permeability. Using this material as a spacer layer, it is possible to achieve a high impedance surface over a broad bandwidth. The proposed structure is different from Sievenpiper's high impedance surface because it has no need for a capacitive layer. As a result, however, it does not suppress the propagation of surface wave modes. The proposed structure is compared to another structure with frequency selective surface loaded with a non-Foster element on a simple spacer layer. In particular, the sensitivity of each structure to component tolerances is considered. The proposed structure shows a high impedance surface over broadband frequency but is much more sensitive than the frequency selective surface structure.