Are you using 2007? There is an option when you "Select Data" (by right clicking on the chart) for "Hidden and Empty Cells" (at the bottom right of the pop-up). Select "Connect data points with line"

I believe 2003 had a similar option.