Pandora FMS の歴史
Pandora FMS の背後にある物語
それは、2001年でした。私が 25歳で Linux 好きだった頃です。 その時、私はスペインの AOL でコミュニケーションおよびセキュリティ部門でプロダクト技術者として働いていました。 しかし、私の仕事は主に Checkpoint のファイヤーウォールに関するものでした。そこで、私は Cabletron および Cisco の実際のネットワークについて多く勉強しました。結果、私の仕事は、ネットワーク環境全体の設定および管理になりました。 何百もの Linux、Windows およびハイエンドなネットワーク機器を含むデータセンター全体が対象となりました。 しかしながら、運用グループは HP の Openview を使っており、”マップ” さえもなく全体を表示することができませんでした。
セキュリティと Linux の知識があったことから、私は、ネットワーク、システム全体、論理的な全体のつながりを確認する役割としてチームの中に居場所をつくりました。 複雑なネットワークトポロジーの異機種混在環境を監視できるまともなツールが無い事に気づきました。 私は、主たる通信システム(ルータ、VPN の RAS 接続、ファイヤーウォールなど)のグラフを描画するために、多くの MRTG を使い、RRDTool で使うスクリプトを作っていました。 OVO は設定変更が複雑であると考えたため、使いませんでした。
その後、BBVA で仕事をすることになりました。新しい仕事は、セキュリティとシステムの世界に、より関連するものでした。そして、AIX と Solaris に、より触れることになりました。 会社で使われていた監視ツールは Big4 でした。しかしそれは、非常に断片化されたフレームワークでした。 部門は、通信とシステムの 2つに分かれており、それぞれ異なるツールと異なる手法を使い、異なる考えで動いていました。
セキュリティ上の理由から、私は、私のシステム(ファイヤーウォール、IDS、UNIX環境、Windowsなど)を監視するツールへのアクセスができませんでした。 そのため、私は手元の仕事を終わらせるために新しい手法を模索せざるを得ませんでした。 その時、私はイベント管理・収集(SEM, SIEM)のための、いくつかのパイロット実装のマネージメントをしていました。 私はその経験から多くを学びました。 私は、ファイヤーウォールの技術専門家になりました。後に、それは複雑なネットワークトポロジに対する理解とチャレンジに対する手助けとなります。 そして、今では Pandora FMS のアーキテクチャ設計の一部になっています。
私は最初に Nagios というソフトウエアを導入してみました。 しかし、エージェントの弱さと融通の利かない監視のコンセプトは私には合いませんでした。 その時、状態を得るだけでなくパフォーマンスも重要でした。 監視レポートに異なるソースからのデータを含んだ強力なグラフが必要でした。 しきい値を設定しアラートを生成できるようにするために、関連ツールをすべて比較してみる必要がありました。 また、機能に対して複雑なテストを(環境が無い状態から)実行し、関連システムのすべての情報を収集することも重要でした。
The vast majority of the time there was no direct access to the system that needed monitoring, so I had to install agents that “blindly” contacted me or another pre-established proxy system. Based on my own experience in logs and event collection tools I saw that all agent-based systems crashed due to their rigid requirements; recent Java-based machines or because of binary package distribution. In many production environments, it was impossible to upgrade versions, install packages on the system or install things as the root user. Nagios wasn’t even close to what I needed or imagined. Since then, I’ve found once and again many systems with uptimes that took several years, which of course nobody wanted to even look at.
The father of Pandora FMS: Arena.
The first integrated software concept that satisfied our needs was “Arena” . It was an environment that gathered data sent by some precarious agents written in KSH. This data was not even standarized, it was sent in plain text and separated by dotted lines “—-“. The server was developed in Perl, made by a colleague of the Department of logical security (Juan H.) and included a frontend in PHP that, in fact, was made from some of my first pieces of code written in PHP.
I knew some web programming by then, but my experience was very poor and responded to very old paradigms (CGI Programming in C and rudimentary HTML in Netscape servers). It was also my first contact with MySQL as my only background then had been my work on Sybase databases.
The birth of Pandora FMS
The needs were growing and the architecture and design just didn’t fit. On the other hand my partner -Juan H.- withdrew from the project and I considered starting Arena again from scratch with another design. That day, in 2003, Pandora was born, but it was just a hobby to occupy my idle time, which at that time I had plenty of.
With the help of a partner (Sergio I.), we created the first Windows agent, based on Windows Scripting Host and coded using VBScript. Needless to say that it was the tip of the iceberg to what it would years later become, in fact, it had many initial issues. I developed the server from scratch, using Perl. I had used Perl scripts in the past to develop small, closely related systems, audit tools, brute force scripts, network scanning, and auditing.
Therefore part of these codes were included in Pandora, and Goliath as well, a WEB engine that makes synthetic transactions. But still, my experience with Perl at that time was limited.
Slowly, and with the help of some friends we created something with documentation to get by, and at the time I was showing the project to co-workers and acquaintances who, thanks to their feedback, helped shape the released version of Pandora FMS. Internally, banks began to see this as an useful tool to get an exact picture of their security systems, especially Checkpoint firewalls and AIX environments. The initial project was called “Pandoramon”.
Pandora FMS as a company
In mid-2005 I proposed to myself living from Pandora professionally, providing network audit services, custom development, software installation, security, etc. At this point the concept for an “Enterprise version” didn’t exist yet. The business model was based on providing services revolving around two products: Pandora and Babel.
Babel was another software with a similar philosophy, oriented to system hardening and the creation of security dashboards. In 2002 I tried to start a business based on software that we developed for Internet-based remote surveillance oriented to security companies (BabyCam), and in 2004, a security consulting company showed interest in it(Ip4All), but none of them were successful.
In summer 2005, David Villanueva -an old workmate- and I decided to create Artica. I left my job as senior specialist consultant at the bank. From this point everything moves faster and faster, Ártica Soluciones Tecnológicas (Technology Solutions) was officially founded in November 2005. The beginning was hard, and Pandora meant nothing but a “cool story” to my first customers. I decided to observe their needs and learn from real-world problems in different environments, always linked to the computer security environment.
One of the first things we did was oficially rename the project to Pandora. In late 2006 we added the tagline “FMS” because only “Pandora” was just too generic of a name. Someone gave us wise words one time: a Free for Free Monitoring System was a very bad way of trying to sell, so for F, we chose the main quality of Pandora, which was closely related to our original logo (an octopus): F for Flexible. If you haven’t seen it, all free software projects tend to choose an animal, ours was an octopus. You can find more information about the story behind that choice on our blog
During the first years, Pandora had a very slow development, since it was not software as such, but rather a purely “Free” project that was useful for allowing us to participate in security consulting projects. At the same time we were also developing Babel Enterprise, and it wasn’t until early 2006 that the first programmer was added to the staff: Esteban Sanchez. He began working ful-time for Pandora. During the early years, Esteban and I were the first programmers, until November 2007, when Ramon Novoa was added to the team. It was both Ramón and Esteban who began to make Pandora a real, publicly available software product. At this time we had established collaborations with different people, of which some of the most important were a New Yorker (Guruevi) and many people from South America, New Zealand and Japan.
Pandora FMS today
There have been many versions, patches, and updates, which during the last 12 years have transformed what was originally a project from a simple technician who strived to solve technical problems into something much larger.
Thanks to the collaboration of dozens of programmers, system technicians, users, and customers, today Pandora FMS is a professional tool for monitoring and managing large environments.
As a “Father” of this creature, I can only say: Thank you all for bringing us here! 🙂
Summary versions of Pandora FMS
First public release: Pandora v0.8.1 (August 2004)
Until August 2004 we didn’t release our first version publicly, which was version 0.8. At that time the development team was composed of three people: Sergio I. bearing part of the initial agent VBScript for Windows, Raul M. helping on different code unrelated tasks, and myself, all working alongside each other to make this possible.
We had to rename the project “Pandoramon” on Sourceforge at the time (because Pandora was already taken).
The first version included features related strictly to agents (We still hadn’t included any kind of remote monitoring) and its interface was pretty basic, to say the least. It didn’t even have an ‘events’ feature, although the graphs were pretty decent. The data model is perhaps the part of the original which has gone through the least changes, but there’s no remaining trace of the original code.
It wasn’t until version 1.0 that we started using a VCS (Version Control System). And though this may sound shocking, the fact is that all of us involved in the project were coming from Networks and Systems, so we were quite the amateurs when it came to Software Development at the time.
Pandora v1.0 (October 2004)
In this version “things got serious.” This version added compatibility with user’s ACL systems.
Pandora v1.2 (December 2006)
We started using Subversion to manage the code. The first version of the agent was written in C ++ and developed by Esteban S.. The agent we had written in VBScript was discarded. Remote monitoring and SNMP trap collecting was added.
Pandora v1.3 (November 2007)
Reports, combined graphics, recon server and customized reports are combined with the previous features. Module and machine templates are designed for the recon server. There is a significant visual change, which makes Pandora start to be taken seriously.
Pandora v2.x (2008-2009)
In this version we created the ‘Tentacle Protocol’. We introduced an export server. ‘Tentacle’ (a unique file transfer protocol with ports assigned by IANA, 41121 / tcp) marked a before and after in making life easier for users when installing agents.
In version 2.x we made an attempt at creating a server version in C but it didn’t materialize, and we continued working on Perl code. Looking back on this, we can say it was a great decision. The main bottleneck in the server is SQL interaction, not internal operations.
Pandora v3.x (2010-2011)
Here we include the first ‘Enterprise’ version. It includes the possibility to remotely modify agent settings, PDF formatted reports, and some other features. We have customers monitoring environments with over 2000 agents .
Pandora v4.x (2012-2013)
We introduce the policy concepts and the meta console. We start playing in the big leagues, as our largest installation yet had already over 8000 monitored servers. We introduce network servers for the Enterprise version of Pandora, with a much higher performance rate to make network checks much faster than on the Open Source servers (10 to 30 times faster).
Pandora v5.x (2014-2015)
We transition the project from SVN to GIT. We introduce the Satellite Server. Thanks to the satellite we achieved a rate of 500 check-ups per second in some specific environments, quite a big number. Plus, this was done through “headless” monitoring, which means the monitoring process was distributed and had no bidirectional connectivity.