Twenty-year-old striker Jake Robinson was a hat-trick scorer for Brighton in a game watched by 10,616 (598 of them backing the South coast club, who went 15th).
And Jackson added: "Credit to Brighton, who played well and finished very well, but they certainly caught us on a bad day.
"At two down at half-time, it was vital we began the second half positively, but their third goal five minutes in completely killed us."
It was a second Galpharm defeat of the season for Town, who have taken only half the 24 points available so far.
"Our away form has been solid, but other than against Nottingham Forest, we haven't produced anything like what we can at our own ground," continued Jackson.
"It's baffling, but it's something that must be sorted out quickly.
"Having won three league games on the trot to go sixth, we wanted maximum points from the home games against Brighton and Scunthorpe.
"Now we have to make sure we at least get half that target, and it's vital we show our mettle against Scunthorpe."
Former Town defender Brian Laws' side, who won 4-1 at the Galpharm last season, beat Orient 3-1 at Glanford Park on Saturday, with Billy Sharp scoring twice to take his season's haul to 12.
Jackson hasn't ruled out having his own eight-goal top scorer Gary Taylor-Fletcher back in the frame.
The 25-year-old missed a third league game after damaging a knee during the 1-0 win at Bradford.
But he is making good progress after having a protective brace removed, and depending on his reaction to training, could figure in Wednesday's 7.30 friendly at Bradford Park Avenue.
Jackson refused to put any blame on inexperienced duo Aaron Hardy, who continued at right-back in place of the injured Andy Holdsworth, and John McCombe, again preferred to Nathan Clarke at centre-back.
"They are young lads who have done okay. Other players should have led by example, but they didn't," he said.
Asked about he 86th minute dismissal of Adams, who was shown a straight red card after fouling Alex Revell, and now misses four matches because it was a straight red for serious foul play (three games) and his second dismisal of the season (one extra) Jackson responded: "At first sight, I thought it was harsh, and the reaction of the Brighton players in going towards the referee (Neil Swarbrick) didn't help.
"Most challenges of that nature are punished by a yellow card."
Brighton midfielder Adam El-Abd had some sympathy for Adams, who was also dismissed at Bradford at the start of the month.
"I felt for him a bit, because when you're 3-0 down at home you do get a bit frustrated," he said.
"It was getting a bit spicy, and we were starting to think Huddersfield might lean on us a bit, which they did.
"There were one or two late ones before the sending-off, and the referee was well placed, but the straight red card was a bit of a surprise."
