Discovered that you can't embed entities in embeddable objects if the entity is managed.
Haven't tried embedding un-managed entities in embeddable objects, though I imagine that's not a great idea.
The the error message isn't all that obvious (attempting to persist object which already has an id).