Orbit stability

This is my first question. I am a space enthusiast but I don't have a formal understanding of celestial mechanics, so I would like to ask how stable against perturbation an orbit is, given the following assumptions:

• orbiting mass negligible vs massive center of orbit
• no atmosphere, no friction, no space dust in the way
• completely rigid bodies so no tidal concerns
• no solar radiation pressure, no Yarkovsky effect, etc
• no obscure funky disturbances not listed here
• object is in orbit at a given altitude with the proper speed for equilibrium

My perturbation would be a nudge either inwards or outwards, e.g. turning on a propeller aimed at the orbit's center (or away from it) for a small amount of time.

What are the mechanics of this: Will the object settle in another stable orbit or will it slowly drift away / crash into the center?

My naive understanding is that, if I push inwards (thus not changing the tangential speed), the object would move to a closer potential orbit but its speed would be too low and it would spiral down to a crash, while it I push outwards the object would have a too fast tangential speed so it would drift away.

According to my (potentially flawed) reasoning, all orbits are unstable, meaning that you always tend to "diverge away" from that equilibrium, like a marble on top of a hill.