The visitors, after conceding a first innings lead of 171, closed on 199-3 in the second innings, a lead of 28 heading into the final day.
Ross Taylor (11 not out) and nightwatchman Neil Wagner (4 not out) survived to stumps.
Off-spinner Sunil Narine claimed 2-67 while fast bowler Kemar Roach took 1-44 to lead the hosts.
Earlier, the West Indies stretched its overnight 442-6 to 522 before being bowled out on the stroke of lunch.
Narsingh Deonarine compiled 79 while captain Darren Sammy supplied a breezy 50.
Chris Martin led New Zealand with 3-134, while part-time off-spinner Kane Williamson (2-47) and fellow pacer Doug Bracewell (2-96) also pitched in.
New Zealand began brightly as it approached the sizeable deficit in the second session, with Guptill and left-hander Daniel Flynn sharing a resolute opening stand of 47.
Flynn contributed 20 before he was deceived by Narine and plumb leg before on the back foot.
Guptill and McCullum took the tourists to tea on 92-1 but the West Indies celebrated a key breakthrough in the first over after the resumption.
But the joy was shortlived as Guptill, who chipped Roach to short extra cover, was reprieved when TV replays confirmed it had been a no ball from the oft-penalized Roach.
Guptill, then on 42, coolly passed his second half-century of the match soon afterwards, off 115 balls, celebrating by driving Roach for a boundary through mid-off.
McCullum also began to blossom, striking off-spinner Marlon Samuels for three fours in an over that cost 18 runs.
He, too, arrived at a fifty, off 92 balls, as the hosts began to look jaded on a flat pitch.
But the West Indies managed to claim both batsmen in the final hour to keep its hopes of victory very much alive.
Guptill, after hitting seven fours off 151 balls in 197 minutes, gloved a big-bouncing Narine off-break to Assad Fudadin at short leg at 170-2. His second wicket partnership with McCullum was worth 123 in two-and-a-half hours.
McCullum was closing in on a seventh test century when he followed, dragging on a Roach delivery as he tried to force off the back foot. The right-hander cracked eight fours off 139 balls in just over three hours.
In the morning session, Deonarine and Sammy stretched their seventh-wicket stand to 69 as the West Indies consolidated its advantage from day three.
Deonarine struck ten fours and a six in a knock that spanned 129 balls and 224 minutes.
Sammy added four fours and three sixes.
New Zealand finally broke through midway through the morning session and claimed the final four wickets for 25 runs.
Deonarine, in his 13th test, fell short of a maiden century dragging on from the inside edge as he played lazily at Chris Martin at 497-7.
Sunil Narine (4) was run out after a mix-up with Sammy at 502-8, Martin providing the return to wicketkeeper Kruger van Wyk.
Sammy got to his half century in grand style, launching Daniel Vettori over wide long-on for six but fell next ball trying to repeat the stroke. Vettori ran back 15 yards into the leg side to snare a fine return catch. Sammy faced 80 balls in 114 minutes.
Bracewell, armed with the third new ball, wrapped up the innings on the stroke of lunch, gaining an lbw verdict following the use of the Decision Review System against Ravi Rampaul.