The Little Carpathians Mountains include a vineyard region with long tradition that dates back up to the Roman Empire period (and according to some opinions, even earlier). In the late 19th century, it was strongly impacted by the phylloxera epidemic, and the vineyard area has significantly reduced here. Large areas of the former vineyards are covered by forests, which mostly have formed spontaneously, but some of them were also planted. This contribution is focused on the impact of the former vineyard land use on the productivity (aboveground, belowground, and total biomass) of such forest herb layer. Research included also the forests, which occupy rocky mounds formed by rock gathering and their placement on the vineyard borders. There were sampled by following four stands, mostly differed by tree composition, origin, age, and succession stage: (i) up to 100 years old spontaneously formed thermophilous acidophilous oak forest on the former vineyard on the granite substrate, (ii) up to 100 years old spontaneously formed oak-dominated forest on the rocky (granite) mounds (borders between the former vineyards formed by rock gathering), (iii) 40–60 years old planted ash stands on the gneiss bedrock, and (iv) 40–60 years old planted ash-dominated stands on the rocky (gneiss) mounds. According to our results, the former land use modified original relief, where the former vineyards have modified soil profile and new relief forms rocky mounds were created. These mounds with no or just shallow soils are usually much less covered by vegetation, thus production of herb layer biomass is lower here than in the adjacent former vineyards. Moreover, rocky mounds show a higher ratio of synanthropic species and apophytes than the adjacent former vineyards, and same as for ratio of therophytes. The younger the stands on the former vineyards, the higher is the ratio of synanthropic species, apophytes, and therophytes. On the other hand, when estimating the production quantity, the values of herb layer production on the former vineyards are similar to those in natural oak-hornbeam forests found in the Little Carpathians Mts. and the adjacent regions, except for the rocky mounds covered by old oak forests, which are less productive. In other words, the former vineyard land use affects the herb layer production quality rather than quantity.