{
"componentChunkName": "component---src-templates-news-detail-ts",
"path": "/news/56",
"result": {"pageContext":{"next":{"id":55,"attributes":{"feature":true,"title":"We’ve launched on the AWS Marketplace and been recognized as an AWS APN Technology Partner! ","content":"We're excited to kickstart our partnership with AWS! \n\nStarting this week, our products will be [available on the AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0&ref=dtl_prodview-alinfa5ygvic6), and we have officially been recognized as a [Technology Partner](https://partners.amazonaws.com/partners/0018a00001lQQCYAA4) in the AWS Partner Network (APN) global community. \n\nWith this move, we are pleased to team up with AWS to help businesses overcome the challenges and costs of using large amounts of real-world data, managing infrastructure complexities, and achieving their cloud-native transition. \n\nWe're super excited about reaching a broad community of developers through this prominent channel.\n\nBy making DBPlusEngine available free, packaging the popular Apache ShardingSphere project, and providing ShardingSphere for Kubernetes - we're continuing to link data and services simply as well as demonstrating our commitment to open source:\n\n- [SphereEx-DBPlusEngine](https://aws.amazon.com/marketplace/pp/prodview-alinfa5ygvic6) is a distributed computing platform to elastically shard & manage your database on any cloud, built with Apache ShardingSphere at its core. In addition to ShardingSphere's functionality, DBPlusEngine provides features such as autoscaling, traffic governance, enterprise security, high-performance clusters, high-availability clusters, and more. \n\n- [ShardingSphere-Proxy](https://aws.amazon.com/marketplace/pp/prodview-kesvb5m5escpo?sr=0-1&ref_=beagle&applicationId=AWSMPContessa) is a transparent database proxy compatible with MySQL and PostgreSQL, working as a distributed database server to provide data sharding, distributed transactions, read/write splitting, HA, query federation features, and more.\n\n- [ShardingSphere for Kubernetes](https://aws.amazon.com/marketplace/pp/prodview-i34uekoeyemgs?sr=0-2&ref_=beagle&applicationId=AWSMPContessa) uses Helm to install a ShardingSphere-Proxy cluster on Kubernetes and provide HPA and HA capabilities.\n\nThis is the first step for our startup towards promoting the \"[Database Plus](https://www.infoq.com/articles/next-evolution-of-database-sharding-architecture/?itm_source=articles_about_ShardingSphere&itm_medium=link&itm_campaign=ShardingSphere)\" and \"[Database Mesh](https://www.database-mesh.io/index.html)\" development concepts and supporting the world of open source. \n\n\nThis news perfectly demonstrates our hyper-growth mode. In the past 12 months or so, we’ve grown our team, signed multiple new deals & partnerships, and moved to new office spaces. In addition to our AWS partnership, SphereEx has also become a [CNCF member](https://www.cncf.io/about/members/). \n\nWith these partnerships, we look forward to accelerating our growth even further and taking on projects and challenges we love to work on. \n\n\nLearn more about our offering on AWS Marketplace [here](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0).","date":"2022-11-30","author":"SphereEx","excerpt":"We’re now an AWS APN Technology Partner, and available on the AWS Marketplace with our DBPlusEngine, and Apache ShardingSphere & ShardingSphere for Kubernetes packaged by us. \n\nThey offer autoscaling, traffic governance, encryption, data sharding, high availability, and DistSQL (Distributed SQL). Available free, for anyone. \n","createdAt":"2022-11-30T06:55:03.268Z","updatedAt":"2022-12-07T10:37:25.697Z","publishedAt":"2022-11-30T07:08:56.019Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":448,"attributes":{"name":"20221130-143323.png","alternativeText":"20221130-143323.png","caption":"20221130-143323.png","width":2160,"height":828,"formats":{"thumbnail":{"name":"thumbnail_20221130-143323.png","hash":"thumbnail_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":245,"height":94,"size":18.24,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_20221130_143323_1e877e8fe6.png"},"large":{"name":"large_20221130-143323.png","hash":"large_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":1000,"height":383,"size":112.8,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/large_20221130_143323_1e877e8fe6.png"},"medium":{"name":"medium_20221130-143323.png","hash":"medium_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":750,"height":288,"size":76.53,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/medium_20221130_143323_1e877e8fe6.png"},"small":{"name":"small_20221130-143323.png","hash":"small_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":500,"height":192,"size":44.84,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_20221130_143323_1e877e8fe6.png"}},"hash":"20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","size":86.2,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/20221130_143323_1e877e8fe6.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-11-30T06:36:47.535Z","updatedAt":"2022-11-30T06:36:47.535Z"}}},"localizations":{"data":[]}}},"prev":{"id":57,"attributes":{"feature":true,"title":"iQiyi goes cloud-native with Apache ShardingSphere & Database Mesh","content":" \n \nIn May of this year, we at [SphereEx](https://www.sphere-ex.com/en/) proposed the [Database Mesh 2.0](https://medium.com/faun/database-mesh-2-0-database-governance-in-a-cloud-native-environment-ac24080349eb) concept. \n[Database Mesh](https://www.database-mesh.io/) is a dynamic concept that is constantly evolving, focusing on database traffic governance and providing sharding, load balancing, observability, and audit capabilities based on database protocol awareness. These capabilities address some of the traffic governance issues. \nFurthermore, Database Mesh emphasizes the development of database reliability engineering (DBRE), providing easier-to-use and superior database governance capability. \n \n[iQiyi](https://www.crunchbase.com/organization/iqiyi) shares our view for the concept and vision of Database Mesh, which is to **achieve high-performance database expansion while tackling data governance issues in the cloud**. \n\niQiyi expanded [ShardingSphere-JDBC](https://shardingsphere.apache.org/document/current/en/overview/#shardingsphere-jdbc) based on its business requirements and conducted a series of tests combined with [Pisanix](https://github.com/database-mesh/pisanix), for the implementation of the Database Mesh concept. \n \n### How did iQiyi prepare to go cloud-native with Database Mesh & ShardingSphere-JDBC?\n\nWith the expansion in the number of features, products & service offerings and the surge in the number of users, enterprises have diversified the number of promotional activities to engage and retain users (flash sales, events, etc.). In successful cases, this led to a huge amount of traffic putting great pressure on their databases. \n\nAs a result, enterprises encounter database issues such as secondary delays and slow queries, with some operations failing to meet business requirements. \nMicroservices and cloud-native bring new possibilities for the business roll-out process and governance. But with more diversified business scenarios and stovepipe data application schemes, data control tends to be isolated. Tech teams face problems such as difficult technology selection, high costs, and complicated management and control.\n \nParticularly, cloud-native architecture is growing mature, and the relationship between business applications and database infrastructure is changing gradually. iQiyi hopes to grasp this new trend and adopt unified management to expand and update databases, thus supporting more businesses and applications migrating to the cloud.\n \nTo meet the requirements for database performance and availability in cloud environments, iQiyi needed to migrate [ShardingSphere](https://shardingsphere.apache.org/)'s local distributed capability to the cloud. To achieve this, iQiyi was looking for a tool that can unify cloud database traffic access in a cloud-native environment and achieve the unified and efficient management of cloud traffic and data.\n \nWhile investigating and testing Pisanix, a Database Mesh solution provided by SphereEx, iQiyi also redeveloped ShardingSphere-JDBC, to meet the requirements for sharding, load balancing, configuration and storage, and security when accessing businesses to database governance platforms.\n \n#### 1. Preparation: iQiyi transforms ShardingSphere-JDBC\n\nCurrently, iQiyi uses a unified config center to store database connection configuration. [KMS technology](https://kms-technology.com/) is used to encrypt database access configuration and ShardingSphere-JDBC is used to implement sharding and load balancing. The complete architecture is shown below.\n \n\n\n \nWhen businesses are connected to the data governance platform, they apply for relevant connection configuration. After they are transformed and the access information is encrypted via KMS, they are stored in the unified configuration storage center. When the application starts, the transformed ShardingSphere-JDBC fetches the configuration and monitors configuration changes to support hot configuration updates.\n \nBefore the transformation, when there was a need to change the configuration, scale-out sharding clusters, upgrade cluster version, or migrate a database to the cloud, it usually required the release of a new version. Also, the DevOps teams had to design complicated procedures such as switchover, rollback, timing selection, grayscale traffic, and data verification, to account for various scenarios.\n \nAfter the transformation, the customized ShardingSphere-JDBC can support sharding cluster scaling or binding changes when adding or modifying table sharding configuration. In the configuration center, you can perform visual operations to modify configurations or bind clusters, and select the configuration of reload timing. When the SDK receives the latest configurations, it starts asynchronous tasks to close the old connection pool and replace the existing one. This facilitates the smooth migration of read/write traffic and greatly simplifies the migration of data governance capabilities to the cloud environment.\n \niQiyi plans to introduce Pisanix-Proxy by accessing Database Mesh, further sinking data governance capability from [SDK](https://en.wikipedia.org/wiki/Software_development_kit) to Sidecar. \n \n#### 2. Data governance capability with Sidecar and building a unified data governance based on Pisanix\n\nIn the traffic access layer, as cloud-native applications move closer to microservices and Serverless, users need to configure complex routing rules, support multiple application-layer protocols, and ensure service access security and the observability of traffic. In response to these requirements, iQiyi used middleware to manage [Redis](https://redis.io/) and [MySQL](https://www.mysql.com/) at the very beginning.\n\n\n \nAdditionally, SmartJedis provided a unified configuration center to support iQiyi's hybrid cloud deployment. In the unified configuration center, configurations in different environments could be dynamically supported. In a non-mesh environment, a direct connection is adopted; while in a mesh environment, RedisProxy in Envoy is used to manage Redis protocol traffic and support hot updates of connection configuration, avoiding downtime after Redis is moved to the cloud. \n \nIn terms of MySQL, iQiyi's R&D team tested Pisanix, the specific implementation of Database Mesh. Written in [Go](https://go.dev/) and [Rust](lhttps://www.rust-lang.org/) for the [Kubernetes](https://kubernetes.io/) environment, Pisanix currently supports [MySQL](https://www.mysql.com/). It includes three components: Pisa-Controller, Pisa-Proxy, and Pisa-Daemon, which provide a local database for users and applications. It supports multi-protocol pluggable architecture, shields the status of real data sources, and provides unified database traffic control capabilities for data DevOps teams.\n\nCurrently, iQiyi still uses ShardingSphere-JDBC to support Java applications. Once Pisanix will be further implemented by iQiyi, the company will implement standardized automatic database maintenance via Pisanix, and achieve the cloud-native orchestration of multiple database governance behaviors by supporting multi-language applications. Based on Database Mesh's standard `CustomResourceDefinition`, such as unified database access declaration configuration and programmable database access resource limitations, iQiyi can rapidly achieve the governance and orchestration of cloud-native databases.\n \n#### 3. iQiyi's plan for Pisanix\n\n**1）Data sharding: achieve high performance on par with ShardingSphere-JDBC in the cloud**\n\nData sharding is an effective way to deal with massive data storage and computing, which is why iQiyi chose Pisanix for cloud-native and non-Java scenarios. Data sharding mainly includes four modules: SQL parser, SQL rewriting, SQL router, and result merger.\n \nTo facilitate the migration of ShardingSphere's powerful local sharding capability to the cloud, Pisanix provides data sharding governance capabilities in the cloud based on the underlying database, allowing users to achieve horizontal scaling computing through Pisanix. At the same time, more custom metrics are available to achieve intelligent, stable, and advanced auto-scaling for Pisa-Proxy.\n \nBased on the Pisa-Controller plane, iQiyi can achieve the management and control of data plane components. Pisa-Proxy can also be combined and deployed in the same Pod with business applications in Sidecar mode to monitor MySQL protocol and obtain the traffic of applications accessing the database. Pisanix also provides iQiyi with a variety of governance capabilities:\n- **SQL traffic governance:** achieve multiple load balancing strategies and current limiting by paring SQL.\n- **Access control:** achieve fine-grained permission control based on the relationship between users and data permission.\n- **SQL firewall:** prevent high-risk SQL from executing.\n- **Observability:** expose various database access metrics such as throughput and latency.\n \nFrom iQiyi's point of view, Pisanix enables the high-performance sharding of both Java and non-Java services in the cloud environment. This achievement lays the foundation for the smooth transition of more businesses.\n \n**2）Read/write splitting: increase database throughput.**\n\nTo improve throughput and availability, many systems adopt a primary-secondary database architecture configuration mode, which is a bit complicated. Therefore, when read requests outnumber write requests, read/write splitting should be used to overcome the performance bottleneck of the database in real-world application scenarios.\n \nRead/write splitting is a widely used technical solution to improve throughput in primary-secondary scenarios, and is capable of improving query performance and reducing server load. It also brings the same problem with data sharding, which makes it more complicated for DevOps teams to operate databases.\n \nCurrently, iQiyi evenly distributes query requests to multiple data copies through the configuration mode of one primary and multi-secondary, which improves the processing capability of the system. This method improves throughput and the availability of the system - even when a database breaks down or a disk is physically damaged, the system can still maintain normal functioning.\n \niQiyi plans to adopt Pisanix's dynamic read/write splitting feature to manage multi-primary and multi-secondary database clusters. After connecting to Pisanix, iQiyi will be able to transparently manage the primary/secondary database with read/write splitting so that users can use the database with the primary/secondary architecture just like a monolithic database.\n \n### Future plan\n\nCurrently, iQiyi has completed its internal transformation for ShardingSphere-JDBC. In the future, it plans to combine Pisanix and ShardingSphere to achieve the unified governance of MySQL. \n\nDriven by the ShardingSphere and Database Mesh communities, Pisanix will continue to develop cloud solutions to meet various usage scenarios, with SphereEx providing reliable technical support for iQiyi and accelerating the transition speed to the cloud. \n \nPisanix is a very young project, which means there are some shortcomings. iQiyi's test shows that Pisanix is limited in its expression support for database and table sharding, and its special configuration for SQL needs to be further improved. \n\nNext, the community will focus on improving Pisanix's online capabilities, including operating status visualization, metrics, circuit breaker degradation strategy, and tracing. \nAdditionally, SQL audit, Pisa-Controller's merge with [Istio](https://istio.io/), and other issues related to compatibility and performance have also been put on the agenda.\n \nIn the coming future, iQiyi will build a MySQL-based unified data access standard and solution based on ShardingSphere-JDBC and Pisanix that is still evolving under the Database Mesh concept. \n\nThrough a unified configuration center and customized Sidecar, iQiyi will gradually make the database access details fully transparent to developers. This way, it can simplify the operating process while enhancing the security of database access, simplifying moving applications moving to the cloud.\n \n----------------------------\nFor more information about Database Mesh and Pisanix, follow the links below:\n[Database Mesh 2.0: Database Governance in a Cloud Native Environment](https://medium.com/faun/database-mesh-2-0-database-governance-in-a-cloud-native-environment-ac24080349eb)\n[Pisanix is Available! An Open Source Database Mesh Solution Launched by SphereEx](https://www.sphere-ex.com/news/43/)\n[ Pisanix GitHub](https://github.com/database-mesh/pisanix)\n","date":"2022-12-16","author":"SphereEx","excerpt":"iQiyi migrates ShardingSphere’s local distributed capability to the cloud with Database Mesh’s Pisanix\n\n","createdAt":"2022-12-15T06:58:49.152Z","updatedAt":"2023-01-16T03:48:29.037Z","publishedAt":"2022-12-16T08:22:35.305Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":457,"attributes":{"name":"iQiyi & Database Mesh.png","alternativeText":"iQiyi & Database Mesh.png","caption":"iQiyi & Database Mesh.png","width":1800,"height":766,"formats":{"thumbnail":{"name":"thumbnail_iQiyi & Database Mesh.png","hash":"thumbnail_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":245,"height":104,"size":23.87,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_i_Qiyi_and_Database_Mesh_dc1577e484.png"},"large":{"name":"large_iQiyi & Database Mesh.png","hash":"large_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":1000,"height":426,"size":142.16,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/large_i_Qiyi_and_Database_Mesh_dc1577e484.png"},"medium":{"name":"medium_iQiyi & Database Mesh.png","hash":"medium_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":750,"height":319,"size":105.4,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/medium_i_Qiyi_and_Database_Mesh_dc1577e484.png"},"small":{"name":"small_iQiyi & Database Mesh.png","hash":"small_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":500,"height":213,"size":63.66,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_i_Qiyi_and_Database_Mesh_dc1577e484.png"}},"hash":"i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","size":134.2,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/i_Qiyi_and_Database_Mesh_dc1577e484.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-12-16T07:14:02.426Z","updatedAt":"2022-12-16T07:14:02.426Z"}}},"localizations":{"data":[]}}},"article":{"id":56,"attributes":{"feature":true,"title":"SphereEx builds a complete enterprise data security ecosystem","content":"Guaranteeing data security is essential, to avoid putting users' personal data at risk of leaks, and avoid damage to enterprises' business security and brand reputation. \n\nAlthough data security cannot generate direct monetary returns, it has become crucial for enterprises. Regulations on data protection have been introduced all over the world, making data security of paramount importance.\n\nFrom an enterprise perspective, the challenges faced in data security can be attributed to both internal and external factors.\n\nEnterprises must speed up building their data security systems since they have a limited amount of time to comply with safety regulations. \n\nLarge enterprises and digital-first companies are typically the focus of regulators when it comes to data security regulation. Additionally, if a company has a presence in the EU for example, it must also adhere to GDPR regulations - making building a data security system of the utmost importance.\n\nHowever, Rome wasn't built in a day. There are many pain points in terms of technology and standards when it comes to data security:\n\n- High business transformation costs: WMS (warehouse management systems) are diversified and large in scale, so application transformation entails high costs. \n\n- High risk during the release phase: there's a high risk when switching applications.\n\n- Switching costs: business switchover is challenging, requiring custom-made strategies.\n\n- Scattered data without unified standards: enterprise data is scattered and without unified authority control.\n\n#### Background\n\n**Data security is positively correlated with business coupling**\n\nAccording to regulations, data related to users' security or commercially sensitive data needs to be encrypted. \n\nHowever, traditional data encryption solutions such as hard disk encryption, file encryption, database TDE encryption, database encryption gateway, and application encryption show a very close positive correlation between their data security and business coupling.\n\n\n\nAs mobile Internet takes over, business adjustments and feature launch frequency are rising, with product features and business scope expanding as well. Enterprises must react rapidly to market demands and sophisticated operating models. \n\nAs a result, traditional stovepipe architecture gradually gave way to microservice architecture - making the \"more secure data leads to more coupled business\" assumption obsolete.\n\nIf we pursue low business coupling, we have to sacrifice a certain degree of data security, which is unacceptable to both the internal requirements of enterprises and external industry standards.\n\n**Business scenarios associated with data encryption**\n\nDepending on specific industry requirements, DevOps teams must maintain a set of encryption and decryption systems for real-world business scenarios. \n\nThe self-maintained encryption system often needs to be rebuilt or modified when the encryption scenario changes. Additionally, for services that have already been launched, it's complicated to transparently and securely implement seamless encryption and transformation without modifying the business logic and SQL.\n\nIn terms of new services, data encryption is required. DevOps teams must achieve data encryption based on encryption requirements since everything is new. Rapid business growth, however, makes it difficult for the original encryption strategy to match the new demands. As a result, large-scale business system transformation is required, causing huge upgrading costs.\n\nFor mature services that are already online and are stored in plain text, when it comes to the the migration and encryption (data cleansing) of the old data and the related business, SQL transformation is required - which is quite complicated. \n\nMoreover, the core business needs to be transformed without impacting the service level. The transformation involves establishing a pre-release environment and coming up with a rollback strategy, which will create significant costs.\n\n### SphereEx-DBPlusEngine: A Comprehensive Data Security Solution\n\nIn response to these issues, SphereEx-DBPlusEngine provides an enterprise cross-platform data security solution for heterogeneous environments requiring zero changes to the original code. \n\nIt also provides online data cleansing, custom algorithms, multiple key management (cloud management is also included), and more, to empower enterprises in coping with various data security requirements.\n\nFollowing the launch of cloud key management, encryption, and online data cleansing features with November's [version 1.2.0 release](https://www.sphere-ex.com/news/50/), SphereEx now completes its data security solution with regulation-compliant testing tools and cryptographic computing in the data flow process, establishing a streamlined enterprise-grade data security system.\n\n---\n\n#### 1. One-Stop Security Compliance\n##### 1.1 Security compliance testing tools\n\nEnterprises must determine which data needs to be encrypted, which comes with its own set of challenges as it is difficult to take into account all the legal and regulatory encryption requirements. \n\nLegal and regulatory encryption requirements are fragmented to say the least, as they vary by location. Nevertheless, enterprises need a tool to quickly determine which data needs to be encrypted.\n\nWith this in mind, we introduced our security compliance testing tools. The tools can examine business data in accordance with national standards and overseas laws and regulations (such as GDPR), and automatically detect the fields in the system that need to be encrypted - reducing negative business impact.\n\n##### 1.2 No-code implementation\n\nWhen it comes to data encryption, enterprises are most concerned about applications being changed. Code changes imply cost, stability, and security concerns as well as many unintended risks.\n\nThe open-source project [ShardingSphere](https://shardingsphere.apache.org) developed a mature no-code implementation capability for data encryption. \n\nThis feature has been enhanced by SphereEx-DBPlusEngine. Enterprises can use SphereEx-DBPlusEngine without changing any application or source code, thus avoiding the business risks caused by code modification. SphereEx-DBPlusEngine enables enterprises to quickly implement data encryption requirements to ensure rapid deployment.\n\n##### 1.3 Key management\n\nAs more businesses are transitioning to the cloud, business data naturally run in cloud environments. However, in a public cloud environment, if enterprises still use the original management method when using SphereEx-DBPlusEngine, hidden dangers in terms of security could manifest:\n\n- Encryption is needed for data storage and use in the cloud, as well as during data transfer.\n\n- The management term of the encryption/decryption key is the entire lifecycle of the data. If the key is lost before the data is destroyed, the data cannot be decrypted.\n\nIn order to address the two issues above, SphereEx-DBPlusEngine offers a cloud-based key management approach by abstracting key management as a standard SPI for cloud vendors like [AWS](https://aws.amazon.com/) and [Alibaba Cloud](https://www.alibabacloud.com/).\n\n\n\nTake AWS as an example. When the program initializes the encryption algorithm, it connects to AWS to retrieve the relevant key stored there and then stores the key in the algorithm. \n\nThe entire data encryption process doesn't include any network communication with the cloud, preventing data flow caused by interaction and fundamentally ensuring key security.\n\nBy offering a cloud-based key management solution, SphereEx provides enterprises with incredibly high key management flexibility and improves the convenience and security of the entire encryption system. \n\nIt can also seamlessly interface with each cloud's key management features to offer the best protection. Moreover, SphereEx-DBPlusEngine supports a number of key management methods to interface with cloud-based, public, and private key management.\n\n##### 1.4 Encrypted data cleansing, backwashing, and rewashing\nWhen enterprises need to migrate new services, they often need to encrypt a large amount of new business data to comply with regulations and internal compliance requirements in terms of data security. A traditional encryption method would not only increase the workload but also delay the entire migration process, affecting the business deployment process.\n\nCurrently, DBPlusEngine already provides an encryption solution. For new tables and services, we can directly configure them using encryption rules; but for existing data tables, the plaintext fields in these tables should be cleaned and converted to encrypted content.\n\n\n\nThe data cleansing job is triggered by [DistSQL](https://shardingsphere.apache.org/document/5.1.0/en/concepts/distsql/). Once the program receives the request, it will create a data cleansing job according to the current data cleansing rule and encryption rule. \n\nThe job is divided into two sections: the query and update tasks:\n\n- The query task is responsible for querying the user's table data and retrieving the plaintext fields that need to be encrypted and then pushing them to the channel. \n\n- The update task obtains the data from the channel, encrypts it, and updates it. \n\nThe whole task creation and execution process interacts with the governance center, allowing users to query its progress or clean up the job through DistSQL.\n\nFurthermore, in an OLAP scenario, DevOps teams cannot analyze the encrypted data, while the business must maintain the encrypted state. \n\nIn this context, the decrypt() function can be used to obtain the plaintext data directly without having to backwash the data, allowing your teams to analyze the ciphertext data and obtain the data value.\n\nSphereEx-DBPlusEngine also supports backwashing and rewashing in the following two scenarios:\n- **Backwashing for business data rollback**\n\nIf some data does not need to be encrypted once the business goes online, or when data masking is performed on data that has been encrypted in large batches, it is necessary to backwash the encrypted data and uniformly convert it to plaintext again.\n\n- **Rewashing for key replacement**\n\nIf the key needs to be changed on a regular basis or at a critical point to ensure long-term data security, it is necessary to backwash the encrypted data, convert it to plaintext, and re-encrypt the data using the new encryption method.\n\n#### 2. Compatibility & Flexibility\n##### 2.1 Flexible encryption algorithm\nSphereEx-DBPlusEngine supports complete data lifecycle security management, with particular attention to the encryption capability for data storage security. It is possible to store and access encrypted data without modifying the application side by implementing data encryption on the client.\n\nSphereEx-DBPlusEngine provides customization capabilities in terms of key management methods and support for [IDEA](https://en.wikipedia.org/wiki/International_Data_Encryption_Algorithm) and other encryption algorithms to meet the wide range of data encryption needs.\n\nTo further increase the efficiency of encrypted storage and computing, SphereEx-DBPlusEngine can work with security hardware for complete and high-performance encryption. It can also provide standard security equipment with integrated hardware and software, further lowering the user's threshold.\n\n##### 2.2 Fine-grained encryption capability\nSphereEx-DBPlusEngine supports multi-dimensional and fine-grained data encryption capability, which can implement data encryption at the row and column levels, and then support data encryption at both the user and tenant levels. \n\nAccording to encryption granularity, different encryption algorithms and key management can be flexibly configured to achieve accurate and adaptable data security protection.\n\n##### 2.3 Suitable for private, public, and hybrid cloud environments deployment\n\nTo increase data security, many enterprises distribute all their data across various environments. This is especially true for industries or application scenarios that have strict requirements for data security. They often need to take into account their diverse deployment environments and complex data security environments.\n\nSphereEx-DBPlusEngine can be flexibly deployed in private, public, and hybrid cloud environments to meet various users' needs. Its key management, compliance detection, data cleansing, fine-grained encryption, encryption algorithm adaptation and other capabilities fully satisfy users' needs for data security in hybrid environments, while shielding the differences created by different underlying environments and ensuring a consistent user experience.\n\n### About SphereEx-DBPlusEngine\n\nSphereEx-DBPlusEngine, a database enhancement engine, adopts a pluggable architecture with functional modularity. In addition to data storage, it also provides data sharding, distributed transactions, data security, and other database application architecture enhancement capabilities.\n\nIn November, SphereEx-DBPlusEngine's version [V1.2.0](https://www.sphere-ex.com/news/50/) was released, adding cloud-based key management and data cleansing capabilities for data security. \n\nIt provides enterprises with comprehensive and powerful compliance testing tools, cloud-based key management, encryption and decryption, and cryptographic computing capabilities, further enhancing the data security protection capability of SphereEx-DBPlusEngine.\n\nTo find out more or request a free trial for DBPlusEngine, you can sign up on our website [here](https://www.sphere-ex.com/account/#/login/signIn?redirect=%2F).\n\nAlternatively, if you are an AWS user, you can learn more about our offering on AWS Marketplace [here](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0).\n\n\n","date":"2022-12-07","author":"SphereEx","excerpt":"SphereEx-DBPlusEngine provides an enterprise cross-platform data security solution for heterogeneous environments requiring zero changes to the original code.","createdAt":"2022-12-07T10:36:49.258Z","updatedAt":"2022-12-07T10:58:34.575Z","publishedAt":"2022-12-07T10:58:34.571Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":452,"attributes":{"name":"20221207-184319.png","alternativeText":"20221207-184319.png","caption":"20221207-184319.png","width":2160,"height":828,"formats":{"thumbnail":{"name":"thumbnail_20221207-184319.png","hash":"thumbnail_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":245,"height":94,"size":48.02,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_20221207_184319_85657aa0ba.png"},"large":{"name":"large_20221207-184319.png","hash":"large_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":1000,"height":383,"size":597.12,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/large_20221207_184319_85657aa0ba.png"},"medium":{"name":"medium_20221207-184319.png","hash":"medium_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":750,"height":288,"size":375.15,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/medium_20221207_184319_85657aa0ba.png"},"small":{"name":"small_20221207-184319.png","hash":"small_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":500,"height":192,"size":186.88,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_20221207_184319_85657aa0ba.png"}},"hash":"20221207_184319_85657aa0ba","ext":".png","mime":"image/png","size":756.49,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/20221207_184319_85657aa0ba.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-12-07T10:44:03.480Z","updatedAt":"2022-12-07T10:44:03.480Z"}}},"localizations":{"data":[]}}},"newsRecommend":[{"id":64,"attributes":{"feature":true,"title":"SphereEx-DBPlusEngine 1.4.0 Release: Enhanced Data Governance for Enterprises","content":"SphereEx-DBPlusEngine 1.3.0, released in January, received a lot of attention and feedback from users and customers. To meet their needs and focus on the industry's future development, we developed a release schedule based on feedback and are excited to announce the release of SphereEx-DBPlusEngine 1.4.0. This version includes significant updates that optimize the migration process and enhance security features such as in-process audit, archiving, and DDL consistency.\n\n# New Capabilities Meet Core User Demands\n\n## 1. Archiving: Support for automatic deletion of expired data\n\nArchiving expired data is a common requirement, and SphereEx-DBPlusEngine 1.4.0 now supports the automatic deletion of expired data. By defining a specific time field in the data table, the system can automatically delete data stored in the table that exceeds the specified time. This simple configuration process saves time and improves efficiency while also enhancing security.\n\n## 2. DDL consistency: Support for consistent execution of DDL statements\n\nDDL consistency has always been a problem in a distributed database environment, especially with sharded tables stored in multiple storage nodes. SphereEx-DBPlusEngine 1.4.0 now supports consistent execution of DDL statements, which can be combined with locks to ensure consistency. This guarantees the maximum concurrency of SQL execution and enhances efficiency.\n\n# Optimized Design Makes Products More Stable and Efficient\n\n## 1. Enhanced migration process: Support for migration of multiple tables with one DistSQL task\n\nTraditional data migration processes execute multiple table tasks in parallel to improve efficiency. However, each DistSQL task can only migrate one table, resulting in low efficiency and manual configuration requirements. SphereEx-DBPlusEngine 1.4.0 now supports the migration of multiple tables with one DistSQL task, greatly improving efficiency. Users only need to specify the tables that need to be migrated in the task configuration.\n\n## 2. Enhanced security features: Support for in-process audit\n\nIn the field of database technology, security has always been a significant issue. SphereEx-DBPlusEngine 1.4.0 introduces support for in-process audit, an important security mechanism that audits relevant SQL operations during execution. With built-in algorithms, it can audit relevant data changes and support two audit actions: log alarm and immediate fuse. This feature enhances system security and allows users to audit and trace more effectively.\n\n# About SphereEx-DBPlusEngine\n\nSphereEx-DBPlusEngine is a product based on the open-source kernel ShardingSphere, providing enterprise-level enhanced data services such as data sharding, distributed transactions, and data security for businesses. It consists of two products: SphereEx-DBPlusEngine-Driver and SphereEx-DBPlusEngine-Proxy, both of which can be deployed independently and support hybrid deployment. They provide standardized data horizontal scaling, distributed transactions, and distributed governance, and can be applied to a variety of diversified scenarios such as Java, homogeneous/heterogeneous languages, cloud-native, etc.\n\nThe release of SphereEx-DBPlusEngine 1.4.0 provides users with a series of new core features and major updates, further expanding the scope of application scenarios for SphereEx-DBPlusEngine and improving efficiency in practical environments. Choosing SphereEx-DBPlusEngine 1.4.0 can not only provide a more stable, efficient, and secure data governance solution but also offer more professional and complete enterprise-level services, reducing costs while accelerating digital business innovation.\n\n# Free Trial\n\nSphereEx-DBPlusEngine 1.4.0 is now available for free download and trial on the [SphereEx official website](https://www.sphere-ex.com/account/#/login/signIn?redirect=%2Fdownload). We are also an AWS APN technical partner, and every version of SphereEx-DBPlusEngine is packaged and released on AWS Marketplace. AWS users are welcome to find our offerings on [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0) to download and experience SphereEx-DBPlusEngine and unlock the convenience and efficiency of cloud-based digital services.","date":"2023-04-03","author":"SphereEx","excerpt":null,"createdAt":"2023-04-04T05:51:34.314Z","updatedAt":"2023-04-04T05:51:36.981Z","publishedAt":"2023-04-04T05:51:36.977Z","locale":"en","newsType":{"data":null},"cover":{"data":null},"localizations":{"data":[]}}},{"id":63,"attributes":{"feature":true,"title":"SphereEx-DBPlusEngine V1.3.0: Enhanced Functionality for Better Data Governance","content":"SphereEx is excited to announce the release of SphereEx-DBPlusEngine V1.3.0, which comes with significant updates to provide enterprises with enhanced data service capabilities. Here are the key improvements in this release:\n\n- Data migration and horizontal scaling: The new update allows users to implement Proxy-based clustering migration, which enhances the computing and data processing capacity of the cluster. Additionally, DBPlusEngine simplifies the operational steps for data migration, making it easier for users.\n- Data masking: The latest update now features mask rules that deform and blur key data when certain key information queries are made, ensuring the safety of sensitive data.\n- Single table to shard table: The Scaling tool in the new update allows the automatic conversion of a single table to a shard table, simplifying users' operations.\n- Driver log collection and metrics monitoring: The DBPlusEngine-Driver form now has the same capability as DBPlusEngine-Proxy, which allows for the visual display of logs and monitoring metrics collected on the Driver side, improving users' experience in log retrieval and monitoring scenarios.\n- Fuzzy query calculations: The new implementation of like calculations in encrypted scenarios supports fuzzy query calculations.\n- Encrypted/decryption data cleaning File Transfer Protocol: In cases of an interruption/failure during encrypted data cleaning, DBPlusEngine can continue to start the data cleaning task based on File Transfer Protocol.\n\n## SphereEx-DBPlusEngine-Mate: Fully Supporting All DBPlusEngine Capabilities\n\nIn addition to the updates to SphereEx-DBPlusEngine, the DBPlusEngine-Mate has come to the v0.3.0 version, which supports all DBPlusEngine capabilities in cloud environments. This allows users to use DBPlusEngine out of ZooKeeper in a Kubernetes environment to get closer to cloud-native ways of managing DBPlusEngine.\n\nThe DBPlusEngine-Mate is a metadata management tool in cloud-native scenarios that seamlessly integrates governance capabilities such as sharding, encrypted data cleaning, read/write separation, and high availability into the Kubernetes metadata system. This allows users to eliminate dependency on ZooKeeper on the cloud, reducing the cost of machine resources and the burden of operations and maintenance staff.\n\nOther Benefits of the DBPlusEngine-Mate v0.3.0 update:\n\n- Guarantees the integrity of DBPlusEngine functionality.\n- Empowers users to fully use DistSQL in the cloud.\n- Optimizes the user experience on the SRE side.\n- Provides SREs and DBAs with the same operational experience as a cloud-native database.\n\n## Experience SphereEx-DBPlusEngine V1.3.0 Today!\n\n\n\n(SphereEx-DBPlusEngine v1.3.0 Product Library page)\n\nSphereEx-DBPlusEngine V1.3.0 is now available for download on the [SphereEx official website](https://www.sphere-ex.com/account/#/login/signIn?redirect=%2Fdownload). Contact our staff to get a limited-time license and experience all the features of SphereEx-DBPlusEngine v1.3.0 for free. AWS users are welcome to find our offerings on the [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0).","date":"2023-02-24","author":"SphereEx","excerpt":null,"createdAt":"2023-04-04T05:47:36.369Z","updatedAt":"2023-04-04T05:47:37.666Z","publishedAt":"2023-04-04T05:47:37.664Z","locale":"en","newsType":{"data":null},"cover":{"data":null},"localizations":{"data":[]}}},{"id":57,"attributes":{"feature":true,"title":"iQiyi goes cloud-native with Apache ShardingSphere & Database Mesh","content":" \n \nIn May of this year, we at [SphereEx](https://www.sphere-ex.com/en/) proposed the [Database Mesh 2.0](https://medium.com/faun/database-mesh-2-0-database-governance-in-a-cloud-native-environment-ac24080349eb) concept. \n[Database Mesh](https://www.database-mesh.io/) is a dynamic concept that is constantly evolving, focusing on database traffic governance and providing sharding, load balancing, observability, and audit capabilities based on database protocol awareness. These capabilities address some of the traffic governance issues. \nFurthermore, Database Mesh emphasizes the development of database reliability engineering (DBRE), providing easier-to-use and superior database governance capability. \n \n[iQiyi](https://www.crunchbase.com/organization/iqiyi) shares our view for the concept and vision of Database Mesh, which is to **achieve high-performance database expansion while tackling data governance issues in the cloud**. \n\niQiyi expanded [ShardingSphere-JDBC](https://shardingsphere.apache.org/document/current/en/overview/#shardingsphere-jdbc) based on its business requirements and conducted a series of tests combined with [Pisanix](https://github.com/database-mesh/pisanix), for the implementation of the Database Mesh concept. \n \n### How did iQiyi prepare to go cloud-native with Database Mesh & ShardingSphere-JDBC?\n\nWith the expansion in the number of features, products & service offerings and the surge in the number of users, enterprises have diversified the number of promotional activities to engage and retain users (flash sales, events, etc.). In successful cases, this led to a huge amount of traffic putting great pressure on their databases. \n\nAs a result, enterprises encounter database issues such as secondary delays and slow queries, with some operations failing to meet business requirements. \nMicroservices and cloud-native bring new possibilities for the business roll-out process and governance. But with more diversified business scenarios and stovepipe data application schemes, data control tends to be isolated. Tech teams face problems such as difficult technology selection, high costs, and complicated management and control.\n \nParticularly, cloud-native architecture is growing mature, and the relationship between business applications and database infrastructure is changing gradually. iQiyi hopes to grasp this new trend and adopt unified management to expand and update databases, thus supporting more businesses and applications migrating to the cloud.\n \nTo meet the requirements for database performance and availability in cloud environments, iQiyi needed to migrate [ShardingSphere](https://shardingsphere.apache.org/)'s local distributed capability to the cloud. To achieve this, iQiyi was looking for a tool that can unify cloud database traffic access in a cloud-native environment and achieve the unified and efficient management of cloud traffic and data.\n \nWhile investigating and testing Pisanix, a Database Mesh solution provided by SphereEx, iQiyi also redeveloped ShardingSphere-JDBC, to meet the requirements for sharding, load balancing, configuration and storage, and security when accessing businesses to database governance platforms.\n \n#### 1. Preparation: iQiyi transforms ShardingSphere-JDBC\n\nCurrently, iQiyi uses a unified config center to store database connection configuration. [KMS technology](https://kms-technology.com/) is used to encrypt database access configuration and ShardingSphere-JDBC is used to implement sharding and load balancing. The complete architecture is shown below.\n \n\n\n \nWhen businesses are connected to the data governance platform, they apply for relevant connection configuration. After they are transformed and the access information is encrypted via KMS, they are stored in the unified configuration storage center. When the application starts, the transformed ShardingSphere-JDBC fetches the configuration and monitors configuration changes to support hot configuration updates.\n \nBefore the transformation, when there was a need to change the configuration, scale-out sharding clusters, upgrade cluster version, or migrate a database to the cloud, it usually required the release of a new version. Also, the DevOps teams had to design complicated procedures such as switchover, rollback, timing selection, grayscale traffic, and data verification, to account for various scenarios.\n \nAfter the transformation, the customized ShardingSphere-JDBC can support sharding cluster scaling or binding changes when adding or modifying table sharding configuration. In the configuration center, you can perform visual operations to modify configurations or bind clusters, and select the configuration of reload timing. When the SDK receives the latest configurations, it starts asynchronous tasks to close the old connection pool and replace the existing one. This facilitates the smooth migration of read/write traffic and greatly simplifies the migration of data governance capabilities to the cloud environment.\n \niQiyi plans to introduce Pisanix-Proxy by accessing Database Mesh, further sinking data governance capability from [SDK](https://en.wikipedia.org/wiki/Software_development_kit) to Sidecar. \n \n#### 2. Data governance capability with Sidecar and building a unified data governance based on Pisanix\n\nIn the traffic access layer, as cloud-native applications move closer to microservices and Serverless, users need to configure complex routing rules, support multiple application-layer protocols, and ensure service access security and the observability of traffic. In response to these requirements, iQiyi used middleware to manage [Redis](https://redis.io/) and [MySQL](https://www.mysql.com/) at the very beginning.\n\n\n \nAdditionally, SmartJedis provided a unified configuration center to support iQiyi's hybrid cloud deployment. In the unified configuration center, configurations in different environments could be dynamically supported. In a non-mesh environment, a direct connection is adopted; while in a mesh environment, RedisProxy in Envoy is used to manage Redis protocol traffic and support hot updates of connection configuration, avoiding downtime after Redis is moved to the cloud. \n \nIn terms of MySQL, iQiyi's R&D team tested Pisanix, the specific implementation of Database Mesh. Written in [Go](https://go.dev/) and [Rust](lhttps://www.rust-lang.org/) for the [Kubernetes](https://kubernetes.io/) environment, Pisanix currently supports [MySQL](https://www.mysql.com/). It includes three components: Pisa-Controller, Pisa-Proxy, and Pisa-Daemon, which provide a local database for users and applications. It supports multi-protocol pluggable architecture, shields the status of real data sources, and provides unified database traffic control capabilities for data DevOps teams.\n\nCurrently, iQiyi still uses ShardingSphere-JDBC to support Java applications. Once Pisanix will be further implemented by iQiyi, the company will implement standardized automatic database maintenance via Pisanix, and achieve the cloud-native orchestration of multiple database governance behaviors by supporting multi-language applications. Based on Database Mesh's standard `CustomResourceDefinition`, such as unified database access declaration configuration and programmable database access resource limitations, iQiyi can rapidly achieve the governance and orchestration of cloud-native databases.\n \n#### 3. iQiyi's plan for Pisanix\n\n**1）Data sharding: achieve high performance on par with ShardingSphere-JDBC in the cloud**\n\nData sharding is an effective way to deal with massive data storage and computing, which is why iQiyi chose Pisanix for cloud-native and non-Java scenarios. Data sharding mainly includes four modules: SQL parser, SQL rewriting, SQL router, and result merger.\n \nTo facilitate the migration of ShardingSphere's powerful local sharding capability to the cloud, Pisanix provides data sharding governance capabilities in the cloud based on the underlying database, allowing users to achieve horizontal scaling computing through Pisanix. At the same time, more custom metrics are available to achieve intelligent, stable, and advanced auto-scaling for Pisa-Proxy.\n \nBased on the Pisa-Controller plane, iQiyi can achieve the management and control of data plane components. Pisa-Proxy can also be combined and deployed in the same Pod with business applications in Sidecar mode to monitor MySQL protocol and obtain the traffic of applications accessing the database. Pisanix also provides iQiyi with a variety of governance capabilities:\n- **SQL traffic governance:** achieve multiple load balancing strategies and current limiting by paring SQL.\n- **Access control:** achieve fine-grained permission control based on the relationship between users and data permission.\n- **SQL firewall:** prevent high-risk SQL from executing.\n- **Observability:** expose various database access metrics such as throughput and latency.\n \nFrom iQiyi's point of view, Pisanix enables the high-performance sharding of both Java and non-Java services in the cloud environment. This achievement lays the foundation for the smooth transition of more businesses.\n \n**2）Read/write splitting: increase database throughput.**\n\nTo improve throughput and availability, many systems adopt a primary-secondary database architecture configuration mode, which is a bit complicated. Therefore, when read requests outnumber write requests, read/write splitting should be used to overcome the performance bottleneck of the database in real-world application scenarios.\n \nRead/write splitting is a widely used technical solution to improve throughput in primary-secondary scenarios, and is capable of improving query performance and reducing server load. It also brings the same problem with data sharding, which makes it more complicated for DevOps teams to operate databases.\n \nCurrently, iQiyi evenly distributes query requests to multiple data copies through the configuration mode of one primary and multi-secondary, which improves the processing capability of the system. This method improves throughput and the availability of the system - even when a database breaks down or a disk is physically damaged, the system can still maintain normal functioning.\n \niQiyi plans to adopt Pisanix's dynamic read/write splitting feature to manage multi-primary and multi-secondary database clusters. After connecting to Pisanix, iQiyi will be able to transparently manage the primary/secondary database with read/write splitting so that users can use the database with the primary/secondary architecture just like a monolithic database.\n \n### Future plan\n\nCurrently, iQiyi has completed its internal transformation for ShardingSphere-JDBC. In the future, it plans to combine Pisanix and ShardingSphere to achieve the unified governance of MySQL. \n\nDriven by the ShardingSphere and Database Mesh communities, Pisanix will continue to develop cloud solutions to meet various usage scenarios, with SphereEx providing reliable technical support for iQiyi and accelerating the transition speed to the cloud. \n \nPisanix is a very young project, which means there are some shortcomings. iQiyi's test shows that Pisanix is limited in its expression support for database and table sharding, and its special configuration for SQL needs to be further improved. \n\nNext, the community will focus on improving Pisanix's online capabilities, including operating status visualization, metrics, circuit breaker degradation strategy, and tracing. \nAdditionally, SQL audit, Pisa-Controller's merge with [Istio](https://istio.io/), and other issues related to compatibility and performance have also been put on the agenda.\n \nIn the coming future, iQiyi will build a MySQL-based unified data access standard and solution based on ShardingSphere-JDBC and Pisanix that is still evolving under the Database Mesh concept. \n\nThrough a unified configuration center and customized Sidecar, iQiyi will gradually make the database access details fully transparent to developers. This way, it can simplify the operating process while enhancing the security of database access, simplifying moving applications moving to the cloud.\n \n----------------------------\nFor more information about Database Mesh and Pisanix, follow the links below:\n[Database Mesh 2.0: Database Governance in a Cloud Native Environment](https://medium.com/faun/database-mesh-2-0-database-governance-in-a-cloud-native-environment-ac24080349eb)\n[Pisanix is Available! An Open Source Database Mesh Solution Launched by SphereEx](https://www.sphere-ex.com/news/43/)\n[ Pisanix GitHub](https://github.com/database-mesh/pisanix)\n","date":"2022-12-16","author":"SphereEx","excerpt":"iQiyi migrates ShardingSphere’s local distributed capability to the cloud with Database Mesh’s Pisanix\n\n","createdAt":"2022-12-15T06:58:49.152Z","updatedAt":"2023-01-16T03:48:29.037Z","publishedAt":"2022-12-16T08:22:35.305Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":457,"attributes":{"name":"iQiyi & Database Mesh.png","alternativeText":"iQiyi & Database Mesh.png","caption":"iQiyi & Database Mesh.png","width":1800,"height":766,"formats":{"thumbnail":{"name":"thumbnail_iQiyi & Database Mesh.png","hash":"thumbnail_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":245,"height":104,"size":23.87,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_i_Qiyi_and_Database_Mesh_dc1577e484.png"},"large":{"name":"large_iQiyi & Database Mesh.png","hash":"large_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":1000,"height":426,"size":142.16,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/large_i_Qiyi_and_Database_Mesh_dc1577e484.png"},"medium":{"name":"medium_iQiyi & Database Mesh.png","hash":"medium_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":750,"height":319,"size":105.4,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/medium_i_Qiyi_and_Database_Mesh_dc1577e484.png"},"small":{"name":"small_iQiyi & Database Mesh.png","hash":"small_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":500,"height":213,"size":63.66,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_i_Qiyi_and_Database_Mesh_dc1577e484.png"}},"hash":"i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","size":134.2,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/i_Qiyi_and_Database_Mesh_dc1577e484.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-12-16T07:14:02.426Z","updatedAt":"2022-12-16T07:14:02.426Z"}}},"localizations":{"data":[]}}},{"id":56,"attributes":{"feature":true,"title":"SphereEx builds a complete enterprise data security ecosystem","content":"Guaranteeing data security is essential, to avoid putting users' personal data at risk of leaks, and avoid damage to enterprises' business security and brand reputation. \n\nAlthough data security cannot generate direct monetary returns, it has become crucial for enterprises. Regulations on data protection have been introduced all over the world, making data security of paramount importance.\n\nFrom an enterprise perspective, the challenges faced in data security can be attributed to both internal and external factors.\n\nEnterprises must speed up building their data security systems since they have a limited amount of time to comply with safety regulations. \n\nLarge enterprises and digital-first companies are typically the focus of regulators when it comes to data security regulation. Additionally, if a company has a presence in the EU for example, it must also adhere to GDPR regulations - making building a data security system of the utmost importance.\n\nHowever, Rome wasn't built in a day. There are many pain points in terms of technology and standards when it comes to data security:\n\n- High business transformation costs: WMS (warehouse management systems) are diversified and large in scale, so application transformation entails high costs. \n\n- High risk during the release phase: there's a high risk when switching applications.\n\n- Switching costs: business switchover is challenging, requiring custom-made strategies.\n\n- Scattered data without unified standards: enterprise data is scattered and without unified authority control.\n\n#### Background\n\n**Data security is positively correlated with business coupling**\n\nAccording to regulations, data related to users' security or commercially sensitive data needs to be encrypted. \n\nHowever, traditional data encryption solutions such as hard disk encryption, file encryption, database TDE encryption, database encryption gateway, and application encryption show a very close positive correlation between their data security and business coupling.\n\n\n\nAs mobile Internet takes over, business adjustments and feature launch frequency are rising, with product features and business scope expanding as well. Enterprises must react rapidly to market demands and sophisticated operating models. \n\nAs a result, traditional stovepipe architecture gradually gave way to microservice architecture - making the \"more secure data leads to more coupled business\" assumption obsolete.\n\nIf we pursue low business coupling, we have to sacrifice a certain degree of data security, which is unacceptable to both the internal requirements of enterprises and external industry standards.\n\n**Business scenarios associated with data encryption**\n\nDepending on specific industry requirements, DevOps teams must maintain a set of encryption and decryption systems for real-world business scenarios. \n\nThe self-maintained encryption system often needs to be rebuilt or modified when the encryption scenario changes. Additionally, for services that have already been launched, it's complicated to transparently and securely implement seamless encryption and transformation without modifying the business logic and SQL.\n\nIn terms of new services, data encryption is required. DevOps teams must achieve data encryption based on encryption requirements since everything is new. Rapid business growth, however, makes it difficult for the original encryption strategy to match the new demands. As a result, large-scale business system transformation is required, causing huge upgrading costs.\n\nFor mature services that are already online and are stored in plain text, when it comes to the the migration and encryption (data cleansing) of the old data and the related business, SQL transformation is required - which is quite complicated. \n\nMoreover, the core business needs to be transformed without impacting the service level. The transformation involves establishing a pre-release environment and coming up with a rollback strategy, which will create significant costs.\n\n### SphereEx-DBPlusEngine: A Comprehensive Data Security Solution\n\nIn response to these issues, SphereEx-DBPlusEngine provides an enterprise cross-platform data security solution for heterogeneous environments requiring zero changes to the original code. \n\nIt also provides online data cleansing, custom algorithms, multiple key management (cloud management is also included), and more, to empower enterprises in coping with various data security requirements.\n\nFollowing the launch of cloud key management, encryption, and online data cleansing features with November's [version 1.2.0 release](https://www.sphere-ex.com/news/50/), SphereEx now completes its data security solution with regulation-compliant testing tools and cryptographic computing in the data flow process, establishing a streamlined enterprise-grade data security system.\n\n---\n\n#### 1. One-Stop Security Compliance\n##### 1.1 Security compliance testing tools\n\nEnterprises must determine which data needs to be encrypted, which comes with its own set of challenges as it is difficult to take into account all the legal and regulatory encryption requirements. \n\nLegal and regulatory encryption requirements are fragmented to say the least, as they vary by location. Nevertheless, enterprises need a tool to quickly determine which data needs to be encrypted.\n\nWith this in mind, we introduced our security compliance testing tools. The tools can examine business data in accordance with national standards and overseas laws and regulations (such as GDPR), and automatically detect the fields in the system that need to be encrypted - reducing negative business impact.\n\n##### 1.2 No-code implementation\n\nWhen it comes to data encryption, enterprises are most concerned about applications being changed. Code changes imply cost, stability, and security concerns as well as many unintended risks.\n\nThe open-source project [ShardingSphere](https://shardingsphere.apache.org) developed a mature no-code implementation capability for data encryption. \n\nThis feature has been enhanced by SphereEx-DBPlusEngine. Enterprises can use SphereEx-DBPlusEngine without changing any application or source code, thus avoiding the business risks caused by code modification. SphereEx-DBPlusEngine enables enterprises to quickly implement data encryption requirements to ensure rapid deployment.\n\n##### 1.3 Key management\n\nAs more businesses are transitioning to the cloud, business data naturally run in cloud environments. However, in a public cloud environment, if enterprises still use the original management method when using SphereEx-DBPlusEngine, hidden dangers in terms of security could manifest:\n\n- Encryption is needed for data storage and use in the cloud, as well as during data transfer.\n\n- The management term of the encryption/decryption key is the entire lifecycle of the data. If the key is lost before the data is destroyed, the data cannot be decrypted.\n\nIn order to address the two issues above, SphereEx-DBPlusEngine offers a cloud-based key management approach by abstracting key management as a standard SPI for cloud vendors like [AWS](https://aws.amazon.com/) and [Alibaba Cloud](https://www.alibabacloud.com/).\n\n\n\nTake AWS as an example. When the program initializes the encryption algorithm, it connects to AWS to retrieve the relevant key stored there and then stores the key in the algorithm. \n\nThe entire data encryption process doesn't include any network communication with the cloud, preventing data flow caused by interaction and fundamentally ensuring key security.\n\nBy offering a cloud-based key management solution, SphereEx provides enterprises with incredibly high key management flexibility and improves the convenience and security of the entire encryption system. \n\nIt can also seamlessly interface with each cloud's key management features to offer the best protection. Moreover, SphereEx-DBPlusEngine supports a number of key management methods to interface with cloud-based, public, and private key management.\n\n##### 1.4 Encrypted data cleansing, backwashing, and rewashing\nWhen enterprises need to migrate new services, they often need to encrypt a large amount of new business data to comply with regulations and internal compliance requirements in terms of data security. A traditional encryption method would not only increase the workload but also delay the entire migration process, affecting the business deployment process.\n\nCurrently, DBPlusEngine already provides an encryption solution. For new tables and services, we can directly configure them using encryption rules; but for existing data tables, the plaintext fields in these tables should be cleaned and converted to encrypted content.\n\n\n\nThe data cleansing job is triggered by [DistSQL](https://shardingsphere.apache.org/document/5.1.0/en/concepts/distsql/). Once the program receives the request, it will create a data cleansing job according to the current data cleansing rule and encryption rule. \n\nThe job is divided into two sections: the query and update tasks:\n\n- The query task is responsible for querying the user's table data and retrieving the plaintext fields that need to be encrypted and then pushing them to the channel. \n\n- The update task obtains the data from the channel, encrypts it, and updates it. \n\nThe whole task creation and execution process interacts with the governance center, allowing users to query its progress or clean up the job through DistSQL.\n\nFurthermore, in an OLAP scenario, DevOps teams cannot analyze the encrypted data, while the business must maintain the encrypted state. \n\nIn this context, the decrypt() function can be used to obtain the plaintext data directly without having to backwash the data, allowing your teams to analyze the ciphertext data and obtain the data value.\n\nSphereEx-DBPlusEngine also supports backwashing and rewashing in the following two scenarios:\n- **Backwashing for business data rollback**\n\nIf some data does not need to be encrypted once the business goes online, or when data masking is performed on data that has been encrypted in large batches, it is necessary to backwash the encrypted data and uniformly convert it to plaintext again.\n\n- **Rewashing for key replacement**\n\nIf the key needs to be changed on a regular basis or at a critical point to ensure long-term data security, it is necessary to backwash the encrypted data, convert it to plaintext, and re-encrypt the data using the new encryption method.\n\n#### 2. Compatibility & Flexibility\n##### 2.1 Flexible encryption algorithm\nSphereEx-DBPlusEngine supports complete data lifecycle security management, with particular attention to the encryption capability for data storage security. It is possible to store and access encrypted data without modifying the application side by implementing data encryption on the client.\n\nSphereEx-DBPlusEngine provides customization capabilities in terms of key management methods and support for [IDEA](https://en.wikipedia.org/wiki/International_Data_Encryption_Algorithm) and other encryption algorithms to meet the wide range of data encryption needs.\n\nTo further increase the efficiency of encrypted storage and computing, SphereEx-DBPlusEngine can work with security hardware for complete and high-performance encryption. It can also provide standard security equipment with integrated hardware and software, further lowering the user's threshold.\n\n##### 2.2 Fine-grained encryption capability\nSphereEx-DBPlusEngine supports multi-dimensional and fine-grained data encryption capability, which can implement data encryption at the row and column levels, and then support data encryption at both the user and tenant levels. \n\nAccording to encryption granularity, different encryption algorithms and key management can be flexibly configured to achieve accurate and adaptable data security protection.\n\n##### 2.3 Suitable for private, public, and hybrid cloud environments deployment\n\nTo increase data security, many enterprises distribute all their data across various environments. This is especially true for industries or application scenarios that have strict requirements for data security. They often need to take into account their diverse deployment environments and complex data security environments.\n\nSphereEx-DBPlusEngine can be flexibly deployed in private, public, and hybrid cloud environments to meet various users' needs. Its key management, compliance detection, data cleansing, fine-grained encryption, encryption algorithm adaptation and other capabilities fully satisfy users' needs for data security in hybrid environments, while shielding the differences created by different underlying environments and ensuring a consistent user experience.\n\n### About SphereEx-DBPlusEngine\n\nSphereEx-DBPlusEngine, a database enhancement engine, adopts a pluggable architecture with functional modularity. In addition to data storage, it also provides data sharding, distributed transactions, data security, and other database application architecture enhancement capabilities.\n\nIn November, SphereEx-DBPlusEngine's version [V1.2.0](https://www.sphere-ex.com/news/50/) was released, adding cloud-based key management and data cleansing capabilities for data security. \n\nIt provides enterprises with comprehensive and powerful compliance testing tools, cloud-based key management, encryption and decryption, and cryptographic computing capabilities, further enhancing the data security protection capability of SphereEx-DBPlusEngine.\n\nTo find out more or request a free trial for DBPlusEngine, you can sign up on our website [here](https://www.sphere-ex.com/account/#/login/signIn?redirect=%2F).\n\nAlternatively, if you are an AWS user, you can learn more about our offering on AWS Marketplace [here](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0).\n\n\n","date":"2022-12-07","author":"SphereEx","excerpt":"SphereEx-DBPlusEngine provides an enterprise cross-platform data security solution for heterogeneous environments requiring zero changes to the original code.","createdAt":"2022-12-07T10:36:49.258Z","updatedAt":"2022-12-07T10:58:34.575Z","publishedAt":"2022-12-07T10:58:34.571Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":452,"attributes":{"name":"20221207-184319.png","alternativeText":"20221207-184319.png","caption":"20221207-184319.png","width":2160,"height":828,"formats":{"thumbnail":{"name":"thumbnail_20221207-184319.png","hash":"thumbnail_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":245,"height":94,"size":48.02,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_20221207_184319_85657aa0ba.png"},"large":{"name":"large_20221207-184319.png","hash":"large_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":1000,"height":383,"size":597.12,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/large_20221207_184319_85657aa0ba.png"},"medium":{"name":"medium_20221207-184319.png","hash":"medium_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":750,"height":288,"size":375.15,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/medium_20221207_184319_85657aa0ba.png"},"small":{"name":"small_20221207-184319.png","hash":"small_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":500,"height":192,"size":186.88,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_20221207_184319_85657aa0ba.png"}},"hash":"20221207_184319_85657aa0ba","ext":".png","mime":"image/png","size":756.49,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/20221207_184319_85657aa0ba.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-12-07T10:44:03.480Z","updatedAt":"2022-12-07T10:44:03.480Z"}}},"localizations":{"data":[]}}},{"id":55,"attributes":{"feature":true,"title":"We’ve launched on the AWS Marketplace and been recognized as an AWS APN Technology Partner! ","content":"We're excited to kickstart our partnership with AWS! \n\nStarting this week, our products will be [available on the AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0&ref=dtl_prodview-alinfa5ygvic6), and we have officially been recognized as a [Technology Partner](https://partners.amazonaws.com/partners/0018a00001lQQCYAA4) in the AWS Partner Network (APN) global community. \n\nWith this move, we are pleased to team up with AWS to help businesses overcome the challenges and costs of using large amounts of real-world data, managing infrastructure complexities, and achieving their cloud-native transition. \n\nWe're super excited about reaching a broad community of developers through this prominent channel.\n\nBy making DBPlusEngine available free, packaging the popular Apache ShardingSphere project, and providing ShardingSphere for Kubernetes - we're continuing to link data and services simply as well as demonstrating our commitment to open source:\n\n- [SphereEx-DBPlusEngine](https://aws.amazon.com/marketplace/pp/prodview-alinfa5ygvic6) is a distributed computing platform to elastically shard & manage your database on any cloud, built with Apache ShardingSphere at its core. In addition to ShardingSphere's functionality, DBPlusEngine provides features such as autoscaling, traffic governance, enterprise security, high-performance clusters, high-availability clusters, and more. \n\n- [ShardingSphere-Proxy](https://aws.amazon.com/marketplace/pp/prodview-kesvb5m5escpo?sr=0-1&ref_=beagle&applicationId=AWSMPContessa) is a transparent database proxy compatible with MySQL and PostgreSQL, working as a distributed database server to provide data sharding, distributed transactions, read/write splitting, HA, query federation features, and more.\n\n- [ShardingSphere for Kubernetes](https://aws.amazon.com/marketplace/pp/prodview-i34uekoeyemgs?sr=0-2&ref_=beagle&applicationId=AWSMPContessa) uses Helm to install a ShardingSphere-Proxy cluster on Kubernetes and provide HPA and HA capabilities.\n\nThis is the first step for our startup towards promoting the \"[Database Plus](https://www.infoq.com/articles/next-evolution-of-database-sharding-architecture/?itm_source=articles_about_ShardingSphere&itm_medium=link&itm_campaign=ShardingSphere)\" and \"[Database Mesh](https://www.database-mesh.io/index.html)\" development concepts and supporting the world of open source. \n\n\nThis news perfectly demonstrates our hyper-growth mode. In the past 12 months or so, we’ve grown our team, signed multiple new deals & partnerships, and moved to new office spaces. In addition to our AWS partnership, SphereEx has also become a [CNCF member](https://www.cncf.io/about/members/). \n\nWith these partnerships, we look forward to accelerating our growth even further and taking on projects and challenges we love to work on. \n\n\nLearn more about our offering on AWS Marketplace [here](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0).","date":"2022-11-30","author":"SphereEx","excerpt":"We’re now an AWS APN Technology Partner, and available on the AWS Marketplace with our DBPlusEngine, and Apache ShardingSphere & ShardingSphere for Kubernetes packaged by us. \n\nThey offer autoscaling, traffic governance, encryption, data sharding, high availability, and DistSQL (Distributed SQL). Available free, for anyone. \n","createdAt":"2022-11-30T06:55:03.268Z","updatedAt":"2022-12-07T10:37:25.697Z","publishedAt":"2022-11-30T07:08:56.019Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":448,"attributes":{"name":"20221130-143323.png","alternativeText":"20221130-143323.png","caption":"20221130-143323.png","width":2160,"height":828,"formats":{"thumbnail":{"name":"thumbnail_20221130-143323.png","hash":"thumbnail_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":245,"height":94,"size":18.24,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_20221130_143323_1e877e8fe6.png"},"large":{"name":"large_20221130-143323.png","hash":"large_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":1000,"height":383,"size":112.8,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/large_20221130_143323_1e877e8fe6.png"},"medium":{"name":"medium_20221130-143323.png","hash":"medium_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":750,"height":288,"size":76.53,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/medium_20221130_143323_1e877e8fe6.png"},"small":{"name":"small_20221130-143323.png","hash":"small_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":500,"height":192,"size":44.84,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_20221130_143323_1e877e8fe6.png"}},"hash":"20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","size":86.2,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/20221130_143323_1e877e8fe6.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-11-30T06:36:47.535Z","updatedAt":"2022-11-30T06:36:47.535Z"}}},"localizations":{"data":[]}}}]}},
"staticQueryHashes": []}