Big striker Matthews chose the perfect moment to score only his second goal of the season, picking the pocket of a home defender before rounding the goalkeeper and slotting into the empty net as the semi-final ticked into injury-time.
It had looked like Town would enjoy a comfortable passage through to the final at Fratton Park as they led 2-0 at half-time, but, for the second time in four days, they stuttered when two goals up and the game was level at 2-2 when Matthews struck.
Basingstoke manager Jason Bristow made five changes for the game, shuffling his pack in the midst of an especially busy period.
The game got off to an open start. Gosport got the ball forward quickly and looked to get crosses into the area but it was the impressive Simon Dunn who had the first effort of note, his low shot from the edge of the area forcing home goalkeeper Gareth Barfoot into an early save.
It was Dunn who gave Town the lead with just 10 minutes on the clock, a decent move ending with the midfielder curling a fine shot into the top corner using his weaker right foot.
An even better chance fell to Dan Wooden on 24 minutes, the striker finding himself through on goal from Tim Sills’ flick but shooting wide of the far post.
Having survived these scares, Town doubled their lead just before half-time, following a bizarre sequence of events.
Kezie Ibe was brought down for a penalty, which the striker took himself. Barfoot made a fine save to keep out his shot from 12 yards and did even better to block Jordace Holder-Spooner’s effort on the rebound, only for Sam Pearce to foul Ibe, conceding a second penalty.
This time Holder-Spooner stepped up to take the kick, finding the bottom corner to put daylight between the sides.
The second half got off to a slow start but Gosport dragged themselves back into the game on 58 minutes. Ibe failed to make the most of a promising break and a hopeful ball forward found Rory Williams in space. He sidestepped two defenders before firing low past Wells.
Town seemed to sit back, inviting pressure, though Holder-Spooner went close on the break, his low shot forcing Barfoot into a good low save.
However, an equaliser was very much on the cards – and it duly arrived on 67 minutes, Williams finding the bottom corner from the edge of the area following a corner.
Both sides had chances to get their noses in front before the end. A good block denied Dunn and Adam Green shot too close to Barfoot, while Robbie Rice made a vital challenge to deny Williams his hat-trick and Town just about survived a goalmouth scramble.
However, it was Matthews who struck the decisive blow, sending Town to their first Hampshire Senior Cup final in six years.
Gosport Borough: Barfoot, Molyneaux, Williams (sub Dunford 78min), Brown, Poate, Pearce, Gosney, Scott, Sills (sub Woodward 45min), Wooden, Wilde. Subs (not used): Igoe, Smith, Ashmore.
Basingstoke Town: Wells, Rice, Green, Dunn, Adams (sub Williams 90min), Ray, Holder-Spooner, Jenkinson, Matthews, Ibe (sub Smart 90min), Barrett (sub Soares 78min). Subs (not used): Maylen, Debattista.