The deal will ramp up competition with US rival Oracle by consolidating SAP's lead in business management and narrowing the gap with Oracle in database technology.
The acquisition, which is SAP's biggest in over two years, is in line with the new management's plan to focus on technology innovation to spur growth.
Sybase specialises in mobile 'on-device' software, which is one of the key elements of SAP's growth strategy.
The announcement comes just days after Jim Hagemann Snabe said he and fellow SAP co-chief executive Bill McDermott were looking for an acquisition to bring new technology to the company.
Snabe had indicated that the deal would be of a similar type and size to the $6.1bn deal with Business Objects in 2007, which was SAP's last big acquisition.
SAP aims to expand its position in enterprise software and move into the new areas of mobile software and software on demand.
McDermott said the deal will combine the world's best business software with the world's most powerful mobile infrastructure platform.
"This is a game-changing transaction for SAP and Sybase customers, who will be better able to connect their employees with key functionality and information from anywhere and make it easier for companies to make faster, more informed business decisions in real time", he said.
Snabe said the combination of SAP and Sybase will give users the option of running their operations from leading mobile devices.
"In addition, innovation around Sybase's established database business will pave the way for 'real' real-time analytics", he said.
Sybase is to be run as a fully owned subsidiary, with John Chen remaining in his role as chief executive and chairman. He will also join SAP's management board.
Sybase's share price jumped more than 35% on news of the planned acquisition by SAP.