The 26 year-old defender has put pen to paper on a three-year deal with the option of a further year, joining David Wagner's side for an undisclosed fee.
The defender had a standout campaign for the Cottagers last season, resulting in him being named in both the EFL and PFA’s SkyBet Championship Teams of the Year as Fulham reached the Play-Off semi-finals.
An attack-minded full back, Scott also got on the scoresheet against the Terriers in April's 4-1 home defeat at the John Smith's Stadium
Speaking about the deal, Huddersfield Town boss David Wagner said: “I’m excited to welcome Scott to Huddersfield Town. Our fans will have seen in our games against Fulham last season what a good player he is and I know his abilities will suit our style of play very well.
“Scott is a good physical presence and is a very capable defender, plus he adds a lot going forward. He shows good football intelligence to time his forward runs and can setup goals for teammates and scoring them himself.
“We already have a very good left back in Chris Löwe and a real young talent in Tareiq Holmes-Dennis, but as we enter our first Premier League season it is so important to have competition and the ability to rotate our side, as we did so successfully last season.”