I will admit I am new to homeschool, and my child is very young, so what works for us may not work for your family. I have searched for and found lots of resources on homeschooling for free or at low cost (without jeopardizing your child's education of course.
Right now the free resources we use regularly are
Our local library has an art and story time for children my daughter's age that she really likes and a lot of the books I use with her come from there are well. Also, luckily our local branch has a large selection of homeschooling books, from many different methods.
2- The Internet-
This one is vast, and there are many sites I like, here are a few.
Starfall http://www.starfall.com/ is a phonics early reading site that both my daughter and I like, and in part is the reason she knows her ABCs in capital and lowercase and phonetically before age 2. It is an amazing site and will follow through with her to early reading.
DLTK has lots and lots of printables and coloring pages.
Ambelside Online is an entire K-12 free Charlotte Mason curriculum.
Prekinders is a great site for both resources and ideas.
3. Other Homeschoolers
I have a few links on my blog for other bloggers that homeschool and often they have lessons available at reasonable prices or for free.
Overall, like just about everything else, you can spend lots of money homeschooling on boxed curriculum and the like, but right now we are really enjoying the free resources we have access to and she is learning quickly and well.