Turns out my son is just plain lost on many of the math concepts in his new 4th grade textbook. Last year, when the teacher gave me all the textbooks, she told me they would be replacing the math book and that I could go ahead and keep that one. Thank goodness I did.
I've decided to review last year's book with my son. If there is something he obviously totally understands, we'll skip on to the next chapter. I was really bothered by the fact that he might be behind (doubted my teaching abilities, doubted the homeschooling thing, got over it), but now I think it doesn't matter if he never gets through the 4th grade book this school year, as long as he understands the material we're covering. Unlike public schools, the fact that he gets it is more important to me that just pushing him along into the next grade. Sure he kind of gets it, but that's not sufficient to me. The first week of school was spent with him crying his way through math lessons, me insisting he knew this material, him declaring he was "too dumb" to do it. No more. Since we've gone back to the 3rd grade book, he happily does his lessons, ask questions about what he doesn't understand and he's learning. Whatever I did wrong last year, I've obviously figured it out this time.
Good luck to us, and to the portfolio evaluator understanding my predicament and finding "sufficient progress" in his work.