In Macbeth, how could the table be full is Banquo didn't show up?

In the tragedy "Macbeth" by William Shakespeare, there's a banquet scene where the ghost of Banquo visits Macbeth. The ghost takes Macbeth's chair, leaving Macbeth say that the table is full and there is no seat for him. Banquo was invited to the banquet, so he was suppose to have the seat at the table. So.. if the ghost took Macbeth's seat, couldn't Macbeth take Banquo's seat? The table wouldn't be full then, would it? SO how come it was full if one guest was missing?