I faced a similar problem and found a workaround solution (Liferay 6.1.1. GA2):
Make sure no one but the Owner role (which presumably is the Administrator) has any permission on this last page. Although you still won't be able to delete the page, nobody (but the Admin) will be able to access it, or even know of its existence. Also, the link to the site's Private Pages from Go To will disappear for all users but the Administrator.
Still, I wonder why deletion of the last page is impossible.
Please sign in to flag this as inappropriate.