The finished softness depends on a base the lime can hold. It bonds by soaking into porous mineral material, and acrylic or latex paint and bare drywall paper are not porous enough, so the walls take the primer system named by the product. Fresh drywall typically needs two layers, first a drywall primer and then the mineral or bonding primer, whereas paint in good condition may need only the bonding layer. Tinting the primer toward the finish color keeps thinner spots from flashing white.
A limewashed wall that stays put for years is only possible if what lies under it is solid. Loose, flaking or powdery paint is therefore the first thing to go, or it gets stabilized, since the new layers will fail with it. After that come the routine steps: filling and sanding damage, scuffing shiny enamel, and washing off kitchen grease and dust. Also check the product label, because certain limewash lines are made for masonry alone and do not belong on drywall.