GAIA: Transforming thematic classification into a stand-alone product

GAIA is an advanced thematic classification and automatic text filtering system, based on technology developed by UZEI. It originated from a specific need: one of ISEA's products, Tentu, required an efficient solution for automatically classifying texts by topic.

After achieving the initial goal, we decided to expand its scope and turn GAIA into an independent product, capable of offering its classification services to new clients. To accomplish this, we designed a comprehensive ecosystem that integrates various applications and technological tools, including:

• Web Application: Built with React and hosted on Firebase, this interface allows users to efficiently manage their service keys.
• REST Services: Implemented with Express.js and deployed on Heroku, these services facilitate communication between clients and UZEI's technology.
• Firebase: We leverage Firebase's suite of tools (user authentication, non-relational databases, static storage, and more) to ensure the system's scalability and optimal performance.

With GAIA established as a robust and functional solution, the project continued to evolve, further refining its performance and expanding its impact. If you want to learn more about its development and architecture, you can explore the details here (only available in Spanish).

Optimization of the Thematic Classification System: AI Integration

In the early stages of development, incorporating new topics into the system was a manual and inefficient process. The selection of key terms for each category relied entirely on human effort, limiting both scalability and operational efficiency.

During my studies for a master's degree in Artificial Intelligence, we identified an opportunity to optimize this process using AI. The proposed solution automated the selection of key terms for each category, significantly reducing manual intervention and enhancing the system’s scalability.

The solution: An automated and intelligent system

The new system was designed to go beyond simple keyword selection, integrating a set of tools to optimize the entire process. Its main components include:

1. Text Collector

An automated system that collects news daily from predefined RSS sources and classifies them according to the target categories.

• Preprocessing: Punctuation marks and irrelevant words (such as prepositions and conjunctions) are removed, and texts are lemmatized to extract root words, improving analysis accuracy.
• Category Balancing: Ensures an even distribution of data across categories, preventing biases in the classification model.
• Cloud Storage: The processed data is stored in a cloud infrastructure for later use in model training.

2. Model Training

Once the data was balanced, multiple algorithms were tested to determine the best option. Although training was initially performed manually, the process was optimized to be fast and efficient. Full technical details are available in the attached document at the end of this section.

3. Model Deployment and Implementation

To facilitate the integration of the model with external clients, the following infrastructure was implemented:

• Development of a REST API using Flask to expose the model as a service.
• Containerization of the system with Docker and deployment on a platform with auto-scaling capabilities, ensuring optimal performance under variable workloads.

Impact and Benefits

Thanks to this optimization, manual workload was drastically reduced, and the process of incorporating new topics was significantly accelerated. Now, human intervention is only required for selecting and categorizing RSS sources, while the rest of the process is fully automated.

If you want to learn more about the development and implementation of this project, you can find further details here (only available in Spanish).

Leadership and Coordination of the Development Team at ISEA

After completing the previous development phases, I took on the challenge of coordinating the software development team at ISEA. Currently, the team consists of three developers and one designer, with the goal of establishing an efficient and scalable workflow.

Building the foundation of a strong and efficient team

Before my arrival, ISEA hired developers on a project-by-project basis, resulting in a fragmented ecosystem with applications built in multiple languages and frameworks based on individual preferences. This lack of standardization made software maintenance and evolution increasingly difficult.

One of my first objectives was to standardize development under a single technology stack. We chose Next.js, a decision that allowed us to:

• Consolidate knowledge and specialize the team in a modern and highly demanded technology.
• Facilitate the onboarding of new team members by providing them with a structured learning foundation.
• Create an ideal training environment for internship students, in collaboration with Mondragón University.

Optimizing deployment and maintenance

The lack of a centralized deployment strategy led to inconsistencies and slowed down issue resolution. Each developer used the deployment method that best suited their workflow, making maintenance and debugging more complex.

To solve this, we decided to unify all deployments on Vercel, a platform that was emerging at the time but showed great potential for optimizing the process. This decision brought key benefits:

• We eliminated recurring deployment issues.
• We increased speed and operational efficiency.
• We significantly reduced maintenance costs.

Results and new challenges

With a consolidated team, an optimized technology infrastructure, and an efficient workflow, we embarked on the development of Evidence Box, an ambitious project that marked a turning point in our trajectory.

You can learn more about this project in the next section.

Evidence Box

Evidence Box is a comprehensive system designed for legally valid evidence recording , composed of multiple interconnected applications.

Originally, the development of this system was handled by multiple companies, each responsible for different components: web application, desktop application, VPN, central server, databases, among others. However, due to external circumstances, these companies ended their collaboration, leaving ISEA with the challenge of completing the system.

In addition to the technical challenge, no prior work was shared with us, which forced us to rebuild the system almost from scratch. What initially seemed like an insurmountable obstacle turned into a unique opportunity for learning and innovation. Together with my team, we tackled this challenge with determination, acquiring new skills and mastering technologies that we had not worked with before.

Through a demanding development process, we successfully replicated and improved most of the system, except for the mobile application, which is still being optimized. The result is the following technological architecture:

Key Components and Their Functionality

Within the ISEA ecosystem, the following elements play a crucial role:

• Evidence Generators: The desktop application, developed with Electron.js, and the mobile application, built for Android, record evidence through screen recordings, network traffic logs, user interaction tracking, and a chronological log of events.
• Web Application: Built with Next.js and hosted on Vercel, this platform enables users to subscribe to services, review generated evidence, and download the necessary tools to operate the system.
• Central Server: Developed with Express.js and deployed on AWS, this backend complements the web application, managing tasks such as notifications and OpenVPN certificate administration.
• VPN: Implemented with OpenVPN on AWS, ensuring secure connections and preventing attacks like Man-in-the-Middle, thereby preserving the integrity of generated evidence.
• Firebase: Used for user authentication, file storage, and non-relational database management, providing a scalable and efficient infrastructure.

Additionally, we integrated external services to guarantee the legal validity and integrity of recorded evidence:

• Metaposta: Functions as a "digital safe and mailbox," storing the hash of each recorded piece of evidence, ensuring its integrity with legal validity.
• Certifaction: Utilizes blockchain technology to certify evidence integrity through a digital signature and a timestamping system.
• Camerfirma: Enables official timestamping, ensuring the legal validity of documents generated within the system.

Despite the initial challenges, this project became an unparalleled opportunity to expand our knowledge and capabilities. Without prior experience in many of the technologies used, the team not only managed to rebuild the system from scratch but also optimized it and adapted it to the highest standards of security and usability.

Evidence Box is not just a functional system; it is a reflection of an accelerated learning process, strategic decision-making, and solid technical execution, which has allowed us to consolidate a high-value product for the team.

Reflection

My experience at ISEA has been a journey filled with challenges, learning opportunities, and significant achievements. Over the years, I have had the opportunity to lead technically complex and strategically important projects, such as transforming a service into a marketable product or developing robust systems with legal applications, like Evidence Box. Additionally, I have been responsible for preparing and successfully passing multiple technical audits conducted by the Ministry for Digital Transformation, ensuring compliance with high standards of quality and technology.

What initially seemed like obstacles—such as the lack of standardized technologies or the need to rebuild systems from scratch—turned into opportunities to establish solid methodologies, optimize processes, and foster a continuous learning environment. This experience has strengthened my ability to identify problems, propose innovative solutions, and, most importantly, implement them effectively with a committed team.

Today, I look to the future with confidence, knowing that the skills and lessons learned at ISEA will be invaluable for tackling new challenges in any project. My motivation lies in continuing to grow, exploring new technologies, and leading initiatives that not only add value to organizations but also inspire the teams I collaborate with.