The world line of a nonaccelerating object is a special type of path called a geodesic. Because no net external forces are acting on the object, conservation of momentum implies that the object will follow the shortest path between two points. On a flat, Euclidean surface a geodesic appears as a straight line.
However on a curved surface the shortest distance between two points is no longer a straight line. The warping of spacetime induced by the presence of the planet causes the geodesic to bend inwards.