It sounds a lot like the vacuum hoses! I had a very similar problem and upon changing the hoses (thanks to 5style) my revs stabilized drastically.
The hoses on my car had never been changed, so when I replaced them they had several holes and tears. Have you checked the condition of the vaccum hoses underneath the ICV? Those apparently are also pretty integral to stable revs at idle. GL and hope that helps!