{
"componentChunkName": "component---src-templates-news-detail-ts",
"path": "/news/49",
"result": {"pageContext":{"next":{"id":48,"attributes":{"feature":true,"title":"Pisanix v0.3.0 is Released, Introducing Data Sharding Strategy!","content":"Data sharding is an effective solution to deal with massive data storage and computation. \n\n[Pisanix](https://www.pisanix.io/en/), a database mesh solution sponsored by SphereEx, now provides data sharding governance capability based on the underlying database - allowing users to scale out computing and storage. \n\nStarting from v0.3.0, Pisanix will gradually support data sharding, with this release supporting single database sharding.\n\n### 1. Introducing data sharding\n\nAs shown in Figure 1 below, data sharding mainly consists of SQL parse, SQL rewriting, SQL route, SQL execution, and result merge.\n\n\n\n**Concepts:**\n\n**SQL Parse:** During the sharding process, once the request is received by Pisa-Proxy, it will first go through SQL Parser, and the SQL will be parsed into AST.\n**SQL Rewriting:** After parsing, Pisa-Proxy will rewrite the current SQL statement according to the sharding rules to generate the real SQL statement to be executed.\n**SQL Route:** Pisa-Proxy routes the rewritten SQL statements to the corresponding data source at the backend to execute the SQL statements according to the sharding rules.\n**SQL Execution:** Pisa-Proxy will rewrite the SQL statement and push it down to the back-end real database for execution.\n**Result Merge:** Pisa-Proxy merges the query results and returns them to the client.\n\n### Introducing SQL Rewriting\n\nSQL rewriting is a critical module in data sharding. Pisa-Proxy needs to rewrite the current SQL statement according to the sharding rules to generate the real SQL statement to be executed. SQL rewriting can be of the following types: \n\n#### Identifier Rewriting\n\nIdentifiers to be rewritten include table names, index names, and schema names.\n\nTable name rewriting means the process of finding the location of a logical table in the original SQL and rewriting it to a real table. Table name rewriting is a typical scenario that requires parsing of SQL. For example, if the logical SQL is:\n\n```\nSELECT order_id FROM order.t_order WHERE order_id = 1;\n```\n\n\nSuppose that the shard key of this table is `order_id` and `order_id=1`, and the number of sharding is specified as two, then the SQL statement would be as follows:\n\n```\nSELECT order_id FROM order.t_order_00001 WHERE order_id = 1;\n```\n\nThe following figure shows the data query process: \n\n\n\nTaking data writing as an example, the data insertion process is as follows: \n\n\n\n**Note:** When the SQL rewriting process modifies the identifier to calculate the real table name, it will automatically add the table index according to the sharding rule. The index rule is table name_index, and the index bit is five. For example, the t_order table is overwritten to t_order_00000. Therefore, you need to create the corresponding table name based on the actual business scenario.\n\n#### Column Supplement Rewriting\n\nThere are two cases that need supplement columns in a query statement. In the first case, Pisa-Proxy needs to get the data during the result merge, but the data is not returned by the queried SQL. \n\nIn this case, it mainly applies to GROUP BY and ORDER BY. When merging the results, you need to group and order the field items according to GROUP BY and ORDER BY, but if the original SQL does not contain grouping or ordering items in the selections, you need to rewrite the original SQL. \n\nFor instance, with an instance with the following SQL statement:\n\n```\nSELECT order_id, user_id FROM t_order ORDER BY user_id;\n```\n\nSince user_id is used for sorting, the data of user_id needs to be retrieved in the result merge, the above SQL statement contains the data of user_id, so there is no need to add columns, and modifying the identifier is enough.\n\n```\nSELECT order_id FROM t_order ORDER BY user_id;\n```\n\nThis SQL depends on the user_id for sorting. Therefore, the column must be supplemented. The rewritten SQL is as follows: \n\n```\nSELECT order_id, user_id AS USER_ID_ORDER_BY_DERIVED_00000 FROM t_order_00000 ORDER BY user_id;\n```\n\nThe second case of column supplement is the use of AVG aggregate functions. \n\nIn distributed scenarios, using (avg1 + avg2 + avg3)/3 to calculate the average is incorrect and should be rewritten as (sum1 + sum2 + sum3) /(count1 + count2 + count3). \n\nIn this case, rewriting the SQL containing AVG to SUM and COUNT is required, and recalculating the average when the results are merged. For example: \n\n```\nSELECT AVG(price) FROM t_order WHERE user_id = 1;\n```\n\nThe rewritten SQL is as follows: \n\n```\nSELECT COUNT(price) AS AVG_DERIVED_COUNT_00000, SUM(price) AS AVG_DERIVED_SUM_00000 FROM t_order_00000 WHERE user_id = 1;\n```\n\n### Configuration Description\n\nThis release supports query, update, deletion, and modification of the single database sharding based on a single shard key. The configuration items are as follows:\n\n\n\n**Note:** The broadcast table, binding table, sub-query, table sharding, distributed rules based on expressions, distributed transaction, and cross-database Join will be gradually supported in later versions.\n\nTaking the scenario in Figure 2 and Figure 3 as an example, its corresponding CRD configuration is as follows: \n\n```\n# Declare a VirtualDatabase as a logical database\napiVersion: core.database-mesh.io/v1alpha1\nkind: VirtualDatabase\nmetadata:\n name: test\n namespace: default\nspec:\n services:\n - databaseMySQL:\n db: test\n host: 127.0.0.1\n password: \"root\"\n port: 3306\n user: root\n name: mysql\n trafficStrategy: test\n dataShard: test\n---\n# Declare TrafficStrategy the specified proxy\napiVersion: core.database-mesh.io/v1alpha1\nkind: TrafficStrategy\nmetadata:\n name: test\n namespace: default\nspec:\n loadBalance:\n simpleLoadBalance:\n kind: random\n selector:\n matchLabels:\n source: test\n---\n# Declare DataShard the sharding rules\napiVersion: core.database-mesh.io/v1alpha1\nkind: DataShard\nmetadata:\n name: test\n namespace: default\n labels:\n source: test\nspec:\n rules:\n - tableName: \"t_order\"\n tableStrategy:\n tableShardingAlgorithmName: \"mod\"\n tableShardingColumn: \"id\"\n shardingCount: 2\n actualDatanodes:\n valueSource:\n nodes:\n - value: \"ds001\"\n---\n# Declare DatabaseEndpoint the identified physical database\napiVersion: core.database-mesh.io/v1alpha1\nkind: DatabaseEndpoint\nmetadata:\n labels:\n source: test\n name: ds001\n namespace: default\nspec:\n database:\n MySQL:\n db: test\n host: mysql.default\n password: root\n port: 3306\n user: root\n```\n\n\n### 2. Pisanix v0.3.0 Overview\nThis release has been made possible thanks to 102 merged PRs by the following contributors:\n\n\n\n#### New Features\n- **Pisa-Controller**\n - Support DataShard CRD #326\n- **Pisa-Proxy**\n - Support single database sharding #338\n \n#### Enhancements\n- **Pisa-Controller**\n - Optimized Sidecar injection #234\n - Support generic static read/write splitting rule #251\n- **Pisa-Proxy**\n - Support SHOW STATUS parser #254\n - Introduce CloudWatch Sinker for later audit events #258\n - Support CREATE TABLESPACE parser #259\n - Support CREATE SERVER #261\n - Front MySQL protocol uses tokio codec #263\n - Support dynamic read/write splitting Monitor switch #264\n - Refactor MySQL server runtime #274\n - Experimental SIMD in SQL Parser #278\n \n#### Fixes\n- **Pisa-Proxy**\n - Fix Apple M1 Macbook build failure in some cases #260\n - Fix malformed SQL format in some cases #291\n - Fix abnormal SQL parsing issues #297\n - Fix unexpected AST in some cases #301\n - Fix Pisa-Proxy crash without receiving VirtualDatabase configuration #319\n \n#### Others\n- **Golang-SDK**\n - Move CRD and kubernetes client to github.com/database-mesh/golang-sdk #247\n - Add DataShard CRD and update VirtualDatabase with DataShard as new spec member in Golang-SDK\n- **Docs**\n - Add sharding doc: https://www.pisanix.io/docs/Features/sharding\n - Optimized microservices-demo deployment instruction #318\n - Fork and optimize the code and configuration of github.com/microservices-demo/microservices-demo for a better demo. See: github.com/database-mesh/microservices-demo\n- **Charts**\n - Update controller and proxy mirror to v0.3.0\n \n#### Known Issues\n- DO NOT support SHOW DATABASES and SHOW TABLES\n- DO NOT support observability under sharding mode\n\n### 3. Community\n\n\n\n**Pisanix is an open source implementation of Database Mesh. It builds unified database governance based on Rust, and aims at providing four major development and application experiences: local database, unified configuration management, multi-protocol support, and cloud-native architecture.**\n\nThe community is currently collecting more user scenarios and applications. If you've started or are planning to test Pisanix, please record it in the [issue](https://github.com/database-mesh/pisanix/issues/282) below. \n\nThe community will prioritize the development and optimization of features related to real scenarios. \n\n\n\n【[Download Link](https://github.com/database-mesh/pisanix/releases/tag/v0.3.0)】\n\nThe Pisanix community organizes an online meeting every two weeks. Here are the details to join: \n- [Mailing list](https://groups.google.com/g/database-mesh)\n- [Biweekly meeting - English community](https://meet.google.com/yhv-zrby-pyt) (from February 16, 2022), on Wednesday 9:00 AM PST\n- [Biweekly meeting - Chinese community](https://meeting.tencent.com/dm/6UXDMNsHBVQO) (from April 27, 2022), on Wednesday 9:00 PM GMT+8\n- [Slack](https://databasemesh.slack.com/)\n- [Meeting minutes](https://bit.ly/39Fqt3x)\n\nYou're welcome to join us!","date":"2022-09-28","author":"SphereEx","excerpt":"Data sharding is an effective solution to deal with massive data storage and computation. \n\nPisanix now provides data sharding governance capability based on the underlying database, allowing users to scale out computing and storage.","createdAt":"2022-09-29T08:17:46.414Z","updatedAt":"2022-09-29T10:19:33.554Z","publishedAt":"2022-09-29T08:18:42.213Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":396,"attributes":{"name":"Database Mesh2.png","alternativeText":"Database Mesh2.png","caption":"Database Mesh2.png","width":1080,"height":414,"formats":{"thumbnail":{"name":"thumbnail_Database Mesh2.png","hash":"thumbnail_Database_Mesh2_00ac6eb990","ext":".png","mime":"image/png","width":245,"height":94,"size":20.45,"path":null,"url":"https://official-media.sphere-ex.com/thumbnail_Database_Mesh2_00ac6eb990.png"},"large":{"name":"large_Database Mesh2.png","hash":"large_Database_Mesh2_00ac6eb990","ext":".png","mime":"image/png","width":1000,"height":383,"size":126.15,"path":null,"url":"https://official-media.sphere-ex.com/large_Database_Mesh2_00ac6eb990.png"},"medium":{"name":"medium_Database Mesh2.png","hash":"medium_Database_Mesh2_00ac6eb990","ext":".png","mime":"image/png","width":750,"height":288,"size":89.97,"path":null,"url":"https://official-media.sphere-ex.com/medium_Database_Mesh2_00ac6eb990.png"},"small":{"name":"small_Database Mesh2.png","hash":"small_Database_Mesh2_00ac6eb990","ext":".png","mime":"image/png","width":500,"height":192,"size":52.99,"path":null,"url":"https://official-media.sphere-ex.com/small_Database_Mesh2_00ac6eb990.png"}},"hash":"Database_Mesh2_00ac6eb990","ext":".png","mime":"image/png","size":60.86,"url":"https://official-media.sphere-ex.com/Database_Mesh2_00ac6eb990.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-09-29T08:05:47.915Z","updatedAt":"2022-09-29T08:05:47.915Z"}}},"localizations":{"data":[]}}},"prev":{"id":50,"attributes":{"feature":true,"title":"SphereEx-DBPlusEngine V1.2.0: a database enhancement engine for cloud-native scenarios","content":"We're releasing SphereEx-DBPlusEngine V1.2.0, upgrading SphereEx-DBPlusEngine-Proxy's enterprise-grade features, and releasing DBPlusEngine-Mate, a metadata governance tool for cloud-native scenarios. \n\nSphereEx-DBPlusEngine V1.2.0 represents a breakthrough for SphereEx-DBPlusEngine in cloud-native scenarios and a significant milestone in SphereEx's enterprise product offering. \n\nDBPlusEngine-Mate empowers SphereEx-DBPlusEngine-Proxy to provide database enhancement services with an improved user experience in cloud-native scenarios.\n\n### SphereEx DBPlusEngine-Mate\n\nThanks to the evolution of the cloud-native technology stack of many Internet enterprises, businesses and data are mainly deployed in proprietary [Kubernetes](https://kubernetes.io/) environments. \n\nIn this context, SphereEx DBPlusEngine V1.2.0 is launching SphereEx DBPlusEngine-Mate in an effort to improve compatibility and user experience in cloud-native scenarios for enterprises using the SphereEx Business Suite.\n\nThe following figure showcases our business solution for the Kubernetes environment. SphereEx empowers users to manage DBPlusEngine in a cloud-native manner with higher availability and lower machine costs, delivering a similar operating experience to cloud-native databases for SREs & DBAs.\n\n\n \nThanks to DBPlusEngine-Mate and DBPlusEngine-Sidecar, enterprises no longer have to rely on [ZooKeeper](https://zookeeper.apache.org/) and can reduce machine costs and DevOps work for SREs. We're also introducing DBPlusEngine-Operator to help enterprises better manage DBPlusEngine.\n\n#### Features\n\nDBPlusEngine-Mate includes three components: DBPlusEngine-Operator, DBPlusEngine-Mate, and DBPlusEngine-Sidecar.\n \n- DBPlusEngine-Operator adopts the typical Operator mode and deploys in the format of `CustomResourceDefinition`, guaranteeing the operation process.\n\n- DBPlusEngine-Mate is the \"brain\". It'll process the metadata corresponding to enhancement rules configured through [DistSQL](https://shardingsphere.apache.org/document/5.1.0/en/concepts/distsql/) or [Kubectl](https://kubernetes.io/docs/reference/kubectl/), and push it to each DBPlusEngine in [RPC](https://en.wikipedia.org/wiki/Remote_procedure_call) form.\n\n- DBPlusEngine-Sidecar is the \"translator\" working for each DBPlusEngine Pod. It returns the configurations in PRC form to the upper-level DBPlusEngine as the standard metadata interface.\n \n#### Advantages\n\nDBPlusEngine-Mate excels in improving SREs user experience when working with internal enterprise platforms, while maintaining the functional integrity of DBPlusEngine and providing DistSQL capability. Together they can build a true cloud-native infrastructure for business applications.\n \nThe launch of DBPlusEngine-Mate demonstrates SphereEx DBPlusEngine commitment towards a real cloud-native application rather than an application that only runs on the cloud. \n\n \n### SphereEx-DBPlusEngine-Proxy V1.2.0\n\nWe originally created SphereEx-DBPlusEngine-Proxy by enhancing [ShardingSphere-Proxy](https://shardingsphere.apache.org/document/current/en/quick-start/shardingsphere-proxy-quick-start/) with enterprise-grade features. \n\nIt can provide data sharding, data encryption and other data-enhancement capabilities in addition to standard horizontal data expansion, distributed transaction and distributed governance - making it applicable to a variety of scenarios including heterogeneous languages and cloud-native environments.\n \n#### Enhanced Features in Version 1.2.0: \n\n- **Enhanced data consistency check feature:** asynchronous execution is optmized.\n\n- **Enhanced DistSQL features:** experience a near-native database experience when managing your cluster with DbPlusEngine thanks to DistSQL.\n\n- **Enhanced compatibility features:** improved compatibility with a wider array of database types and SQL.\n\n- **Enhanced visualization feature:** agent configuration refactoring to optimize user experience.\n\n- **Enhanced data security features:** in addition to [AES](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard), more enterprise encryption algorithms are added.\n\n- **Enhanced data security feature:** decryption and data cleaning are added.\n\n- **Enhanced data security feature:** online key changes are supported.\n\n- **New governance center feature:** our self-developed SphereEx-DBPlusEngine-Mate is added to the governance center, allowing you to replace ZooKeeper to manage metadata in the cloud & Kubernetes environments.\n\nIn the future, SphereEx-DBPlusEngine will continue to provide comprehensive and distributed cloud-native database enhancement services based on standard cloud-native architecture. Our enterprise product offering will continue to strive to meet users' requirements in terms of multi-cloud, heterogeneity, encryption, and migration.\n","date":"2022-11-01","author":"SphereEx","excerpt":"We're releasing SphereEx-DBPlusEngine V1.2.0 together with DBPlusEngine-Mate, making the SphereEx product family cloud-native!\n\n\n","createdAt":"2022-11-01T03:30:42.770Z","updatedAt":"2022-11-01T09:16:21.772Z","publishedAt":"2022-11-01T08:01:23.121Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":419,"attributes":{"name":"DBPlusEngine V1.2.0 cover.png","alternativeText":"DBPlusEngine V1.2.0 cover.png","caption":"DBPlusEngine V1.2.0 cover.png","width":1074,"height":460,"formats":{"thumbnail":{"name":"thumbnail_DBPlusEngine V1.2.0 cover.png","hash":"thumbnail_DB_Plus_Engine_V1_2_0_cover_c6d56ec2cb","ext":".png","mime":"image/png","width":245,"height":105,"size":20.5,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_DB_Plus_Engine_V1_2_0_cover_c6d56ec2cb.png"},"large":{"name":"large_DBPlusEngine V1.2.0 cover.png","hash":"large_DB_Plus_Engine_V1_2_0_cover_c6d56ec2cb","ext":".png","mime":"image/png","width":1000,"height":428,"size":165.11,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/large_DB_Plus_Engine_V1_2_0_cover_c6d56ec2cb.png"},"medium":{"name":"medium_DBPlusEngine V1.2.0 cover.png","hash":"medium_DB_Plus_Engine_V1_2_0_cover_c6d56ec2cb","ext":".png","mime":"image/png","width":750,"height":321,"size":105.72,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/medium_DB_Plus_Engine_V1_2_0_cover_c6d56ec2cb.png"},"small":{"name":"small_DBPlusEngine V1.2.0 cover.png","hash":"small_DB_Plus_Engine_V1_2_0_cover_c6d56ec2cb","ext":".png","mime":"image/png","width":500,"height":214,"size":58.48,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_DB_Plus_Engine_V1_2_0_cover_c6d56ec2cb.png"}},"hash":"DB_Plus_Engine_V1_2_0_cover_c6d56ec2cb","ext":".png","mime":"image/png","size":117.61,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/DB_Plus_Engine_V1_2_0_cover_c6d56ec2cb.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-11-01T03:27:25.423Z","updatedAt":"2022-11-01T03:27:25.423Z"}}},"localizations":{"data":[]}}},"article":{"id":49,"attributes":{"feature":false,"title":"We're sponsoring KubeCon+CloudNativeCon NA 2022 and launching DBPlusSuite to bring Database Plus & Database Mesh offerings with a cloud native database governance architecture","content":"We are thrilled to be sponsoring the [CNCF](https://www.cncf.io/) flagship conference [KubeCon + CloudNativeCon NA 2022](https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/) from October 24 - 28, 2022, and to launch DBPlusSuite V1.1!\n\nAs a [CNCF silver member](https://www.cncf.io/about/members/), SphereEx has been following through on its commitment to cloud native. \n\nFrom Database Plus to Database Mesh, from supporting the 5.2.0 release of the popular Big Data project [Apache ShardingSphere](https://faun.pub/apache-shardingsphere-5-2-0-is-released-bringing-new-cloud-native-possibilities-8d674d964a93?source=your_stories_page-------------------------------------) to initiating and sponsoring a new database mesh open source project with [Pisanix 0.3.0](https://www.sphere-ex.com/news/48/), we have been fully committed to expanding cloud native possibilities.\n\nWe believe our sponsorship of KubeCon + CloudNativeCon proves this commitment again.\n\n[SphereEx](https://www.sphere-ex.com/) builds distributed data infrastructures while delivering a SaaS experience through the cloud, and was founded by the core contributor team of [Apache ShardingSphere](https://shardingsphere.apache.org/) - an open-source ecosystem to transform any database into a distributed database system and enhance it with data sharding, elastic scaling, encryption features & more. \n\nWe are now launching an enterprise-grade pluggable data service enhancement platform: SphereEx-[DBPlusSuite V1.1](https://www.sphere-ex.com/), which includes SphereEx-Boot, SphereEx- Console, and SphereEx DBPlusEngine. \n\nThe business suite brings several features including Autoscaling, Traffic Governance, Data Sharding, and DistSQL while remaining multi-cloud, breaking performance records, and maintaining an open source DNA. \n\nOur pioneering [\"Database Plus\" and \"Database Mesh\"](https://www.sphere-ex.com/products/concepts/) concepts that drive our work, are designed to unlock innovative ways to leverage and manage data and improve enterprise data applications.\n\nThanks to Apache ShardingSphere, our offerings have been tested by stock market-listed corporations in over 170 enterprise and production environments covering the financial, logistics, e-commerce, internet & cloud computing fields, to help achieve enterprise digital transformation.\n\n**You can join us virtually at our [KubeCon virtual booth](https://cncf-dot-yamm-track.appspot.com/2-22rLgN3QFsyChEI7yF3ez6C7zcZI5KE-JGAWZuH9oYBcH8jgwGI3wdrF8JY-hjVexkjT7XZlmADl7wsEiRq2F3TrF5qeJnAXwz84mEjzMy_BTgQFfCU7V7xAWPRCEqUv7aBrKvdceE7BFR-XwNlZgqKmBv0_OxkhpDRJenknBYHtJJk2XfgZuAkwLAQRhUlNFPZpGPsJJkwFP-Inh0ne6Iiu6FFW2xQOGFy28vE8nqzI8dfkT-Rkp5E9vwDVTObk2vr9bjfE7Fkbv-VtFc3vsV_5ecnOxRJ3NI6e_AjJqIfShAsblS4ZW7qxlgVcs-wre5T0XlbRPFwGHC4w5X1J7o3AiNE5n8c_GlXdRsmVtcXTXMCMWzdZmuaYxxVzYjuIJ-53A2Op0uuS6hBiurhLS1vFzrDm9JZFc5Clm1GkVTMlyZPksxbDt1mC4CTZYnglxdMLYDioRCJnF07QwIuooQT) to learn more about DBPlusEngine, Database Mesh, Database Plus and Apache ShardingSphere.**\n\n\n\n\n\n### Enterprise-grade data services enhancement engine: SphereEx DB Plus Engine\n\nSphereEx DB Plus Engine is an enterprise data service platform, built with Apache ShardingSphere at its core. In addition to ShardingSphere's functionality, DB Plus Engine provides features such as enterprise security, high-performance clusters, high-availability clusters, and more. It consists of two products, SphereEx DB Plus Engine-Driver and SphereEx DB Plus Engine-Proxy, to be deployed in combination or independently.\n\nThe newly released SphereEx DB Plus Engine-Proxy V1.1 focuses on updating capabilities in encryption and decryption, dual routing, and user privileges.\n\n#### Data features:\n\n• Autoscaling to automatically increase or decrease compute, memory, or networking resources. \n• Traffic governance to intelligently distribute traffic, for higher throughput and efficient query handling. \n• Data sharding to deal with massive data storage and computation. \n• Automatic data encryption and decryption support user-defined data types.\n• Dual routing: JDBC and Proxy integration based on dual routing capability.\n \n#### User privilege management:\n\n• Adds a new login mode: LDAP authentication.\n• Adds role management.\n• Adds table-level privilege control.\n• Adds column-level privilege control.\n• User privileges can be changed online.\n\nBoth SphereEx DB Plus Engine-Driver and SphereEx DB Plus Engine can provide standardized data scale-out, distributed transactions, and distributed governance capabilities - applicable to Java isomorphic & heterogeneous languages scenarios, and cloud-native environments.\n\n\n#### Installation and deployment: SphereEx-Boot\n\nSphereEx-Boot is the ideal solution to install and manage a variety of cluster management tools.\n\nSphereEx-Boot V1.1 is a Python-developed command line tool for easy management of SphereEx DB Plus Engine-Proxy clusters, providing significant usability and compatibility improvements.\n\n• In terms of usability, SphereEx-Boot provides an offline installation mode allowing users to install and deploy products in an offline environment with a single command.\n• In terms of compatibility, SphereEx-Boot supports the installation of Prometheus, allowing for the easy monitoring of SphereEx's product components.\n\nIn the context of the growing [Apache ShardingSphere ecosystem](https://shardingsphere.apache.org/), SphereEx-Boot manages many components within the SphereEx DB Plus Engine ecosystem and acts as a cluster management tool in enterprise production environments.\n \n#### Data management and control: SphereEx-Console\n\nSphereEx-Console, a data visualization platform for the management and control of the SphereEx enterprise data services platform, provides a user-friendly experience. \n\nIt's integrated into a solution including resources, instances, plugins, and more to provide a one-stop user experience with SphereEx DB Plus Engine at its core.\n\nThe newly released SphereEx-Console V1.1 further expands product capabilities in terms of cluster adaptation, [DistSQL (Distributed SQL)](https://shardingsphere.apache.org/blog/en/material/jul_26_an_introduction_to_distsql/), and visualization.\n\n• Cluster management: SphereEx-Console can manage the SphereEx DB-Plus-Engine cluster with one click and seamlessly access users' existing platforms, significantly improving product compatibility.\n\n• DistSQL: SphereEx-Console fully supports DistSQL for its operation, enabling users to create and bind various kinds of tables - greatly improving usability for DBAs.\n\n• Visualization: SphereEx-Console V1.1 enables users to customize monitoring indicators and interface management visualization.\n\nSphereEx-Console, a SphereEx business data services cluster platform centered on managing Apache ShardingSphere, simplifies user configuration, operation, and deployment processes. Lowering entry barriers for beginners while solving enterprise management and control pain points, delivers enhanced operation & management to enterprises.\n\n\n\nOur business suite will continue following the [Database Plus](https://faun.pub/whats-the-database-plus-concepand-what-challenges-can-it-solve-715920ba65aa) design concept for distributed database systems, while the \"cloud\" being our key technology development trend. \n\nToday, cloud native has become essengtial for hosting large-scale applications and mastering the future of technology.\n\nSphereEx will actively adapt to the data needs of cloud native scenarios, bringing cloud native databases closer to users and providing enterprises with flexible and efficient cloud data governance to accelerate their transition from on-premise to cloud.\n\n\n**Official Website:** [https://www.sphere-ex.com/](https://www.sphere-ex.com/)\n**LinkedIn:** https://www.linkedin.com/company/sphere-ex/\n**Twitter:** @Sphere_Ex\n**Apache ShardingSphere:** https://shardingsphere.apache.org\n**Pisanix:** https://www.pisanix.io\n**KubeCon Registration:** https://events.linuxfoundation.org/kubecon-cloudnativecon-north-america/\n**Our KubeCon Booth:** https://tinyurl.com/SphereEx-Booth","date":"2022-10-20","author":"SphereEx","excerpt":"We are thrilled to sponsor the KubeCon + CloudNativeCon NA 2022 and to launch DBPlusSuite V1.1!\nLearn more or connect with us at our KubeCon booth.","createdAt":"2022-10-20T10:07:55.310Z","updatedAt":"2022-10-20T10:28:05.279Z","publishedAt":"2022-10-20T10:08:37.486Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":416,"attributes":{"name":"KubeCon+CloudNativeCon NA.png","alternativeText":"KubeCon+CloudNativeCon NA.png","caption":"KubeCon+CloudNativeCon NA.png","width":652,"height":535,"formats":{"thumbnail":{"name":"thumbnail_KubeCon+CloudNativeCon NA.png","hash":"thumbnail_Kube_Con_Cloud_Native_Con_NA_22ebbb46ed","ext":".png","mime":"image/png","width":190,"height":156,"size":42.48,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_Kube_Con_Cloud_Native_Con_NA_22ebbb46ed.png"},"small":{"name":"small_KubeCon+CloudNativeCon NA.png","hash":"small_Kube_Con_Cloud_Native_Con_NA_22ebbb46ed","ext":".png","mime":"image/png","width":500,"height":410,"size":207.03,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_Kube_Con_Cloud_Native_Con_NA_22ebbb46ed.png"}},"hash":"Kube_Con_Cloud_Native_Con_NA_22ebbb46ed","ext":".png","mime":"image/png","size":219.47,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/Kube_Con_Cloud_Native_Con_NA_22ebbb46ed.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-10-20T09:55:38.000Z","updatedAt":"2022-10-20T09:55:38.000Z"}}},"localizations":{"data":[]}}},"newsRecommend":[{"id":64,"attributes":{"feature":true,"title":"SphereEx-DBPlusEngine 1.4.0 Release: Enhanced Data Governance for Enterprises","content":"SphereEx-DBPlusEngine 1.3.0, released in January, received a lot of attention and feedback from users and customers. To meet their needs and focus on the industry's future development, we developed a release schedule based on feedback and are excited to announce the release of SphereEx-DBPlusEngine 1.4.0. This version includes significant updates that optimize the migration process and enhance security features such as in-process audit, archiving, and DDL consistency.\n\n# New Capabilities Meet Core User Demands\n\n## 1. Archiving: Support for automatic deletion of expired data\n\nArchiving expired data is a common requirement, and SphereEx-DBPlusEngine 1.4.0 now supports the automatic deletion of expired data. By defining a specific time field in the data table, the system can automatically delete data stored in the table that exceeds the specified time. This simple configuration process saves time and improves efficiency while also enhancing security.\n\n## 2. DDL consistency: Support for consistent execution of DDL statements\n\nDDL consistency has always been a problem in a distributed database environment, especially with sharded tables stored in multiple storage nodes. SphereEx-DBPlusEngine 1.4.0 now supports consistent execution of DDL statements, which can be combined with locks to ensure consistency. This guarantees the maximum concurrency of SQL execution and enhances efficiency.\n\n# Optimized Design Makes Products More Stable and Efficient\n\n## 1. Enhanced migration process: Support for migration of multiple tables with one DistSQL task\n\nTraditional data migration processes execute multiple table tasks in parallel to improve efficiency. However, each DistSQL task can only migrate one table, resulting in low efficiency and manual configuration requirements. SphereEx-DBPlusEngine 1.4.0 now supports the migration of multiple tables with one DistSQL task, greatly improving efficiency. Users only need to specify the tables that need to be migrated in the task configuration.\n\n## 2. Enhanced security features: Support for in-process audit\n\nIn the field of database technology, security has always been a significant issue. SphereEx-DBPlusEngine 1.4.0 introduces support for in-process audit, an important security mechanism that audits relevant SQL operations during execution. With built-in algorithms, it can audit relevant data changes and support two audit actions: log alarm and immediate fuse. This feature enhances system security and allows users to audit and trace more effectively.\n\n# About SphereEx-DBPlusEngine\n\nSphereEx-DBPlusEngine is a product based on the open-source kernel ShardingSphere, providing enterprise-level enhanced data services such as data sharding, distributed transactions, and data security for businesses. It consists of two products: SphereEx-DBPlusEngine-Driver and SphereEx-DBPlusEngine-Proxy, both of which can be deployed independently and support hybrid deployment. They provide standardized data horizontal scaling, distributed transactions, and distributed governance, and can be applied to a variety of diversified scenarios such as Java, homogeneous/heterogeneous languages, cloud-native, etc.\n\nThe release of SphereEx-DBPlusEngine 1.4.0 provides users with a series of new core features and major updates, further expanding the scope of application scenarios for SphereEx-DBPlusEngine and improving efficiency in practical environments. Choosing SphereEx-DBPlusEngine 1.4.0 can not only provide a more stable, efficient, and secure data governance solution but also offer more professional and complete enterprise-level services, reducing costs while accelerating digital business innovation.\n\n# Free Trial\n\nSphereEx-DBPlusEngine 1.4.0 is now available for free download and trial on the [SphereEx official website](https://www.sphere-ex.com/account/#/login/signIn?redirect=%2Fdownload). We are also an AWS APN technical partner, and every version of SphereEx-DBPlusEngine is packaged and released on AWS Marketplace. AWS users are welcome to find our offerings on [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0) to download and experience SphereEx-DBPlusEngine and unlock the convenience and efficiency of cloud-based digital services.","date":"2023-04-03","author":"SphereEx","excerpt":null,"createdAt":"2023-04-04T05:51:34.314Z","updatedAt":"2023-04-04T05:51:36.981Z","publishedAt":"2023-04-04T05:51:36.977Z","locale":"en","newsType":{"data":null},"cover":{"data":null},"localizations":{"data":[]}}},{"id":63,"attributes":{"feature":true,"title":"SphereEx-DBPlusEngine V1.3.0: Enhanced Functionality for Better Data Governance","content":"SphereEx is excited to announce the release of SphereEx-DBPlusEngine V1.3.0, which comes with significant updates to provide enterprises with enhanced data service capabilities. Here are the key improvements in this release:\n\n- Data migration and horizontal scaling: The new update allows users to implement Proxy-based clustering migration, which enhances the computing and data processing capacity of the cluster. Additionally, DBPlusEngine simplifies the operational steps for data migration, making it easier for users.\n- Data masking: The latest update now features mask rules that deform and blur key data when certain key information queries are made, ensuring the safety of sensitive data.\n- Single table to shard table: The Scaling tool in the new update allows the automatic conversion of a single table to a shard table, simplifying users' operations.\n- Driver log collection and metrics monitoring: The DBPlusEngine-Driver form now has the same capability as DBPlusEngine-Proxy, which allows for the visual display of logs and monitoring metrics collected on the Driver side, improving users' experience in log retrieval and monitoring scenarios.\n- Fuzzy query calculations: The new implementation of like calculations in encrypted scenarios supports fuzzy query calculations.\n- Encrypted/decryption data cleaning File Transfer Protocol: In cases of an interruption/failure during encrypted data cleaning, DBPlusEngine can continue to start the data cleaning task based on File Transfer Protocol.\n\n## SphereEx-DBPlusEngine-Mate: Fully Supporting All DBPlusEngine Capabilities\n\nIn addition to the updates to SphereEx-DBPlusEngine, the DBPlusEngine-Mate has come to the v0.3.0 version, which supports all DBPlusEngine capabilities in cloud environments. This allows users to use DBPlusEngine out of ZooKeeper in a Kubernetes environment to get closer to cloud-native ways of managing DBPlusEngine.\n\nThe DBPlusEngine-Mate is a metadata management tool in cloud-native scenarios that seamlessly integrates governance capabilities such as sharding, encrypted data cleaning, read/write separation, and high availability into the Kubernetes metadata system. This allows users to eliminate dependency on ZooKeeper on the cloud, reducing the cost of machine resources and the burden of operations and maintenance staff.\n\nOther Benefits of the DBPlusEngine-Mate v0.3.0 update:\n\n- Guarantees the integrity of DBPlusEngine functionality.\n- Empowers users to fully use DistSQL in the cloud.\n- Optimizes the user experience on the SRE side.\n- Provides SREs and DBAs with the same operational experience as a cloud-native database.\n\n## Experience SphereEx-DBPlusEngine V1.3.0 Today!\n\n\n\n(SphereEx-DBPlusEngine v1.3.0 Product Library page)\n\nSphereEx-DBPlusEngine V1.3.0 is now available for download on the [SphereEx official website](https://www.sphere-ex.com/account/#/login/signIn?redirect=%2Fdownload). Contact our staff to get a limited-time license and experience all the features of SphereEx-DBPlusEngine v1.3.0 for free. AWS users are welcome to find our offerings on the [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0).","date":"2023-02-24","author":"SphereEx","excerpt":null,"createdAt":"2023-04-04T05:47:36.369Z","updatedAt":"2023-04-04T05:47:37.666Z","publishedAt":"2023-04-04T05:47:37.664Z","locale":"en","newsType":{"data":null},"cover":{"data":null},"localizations":{"data":[]}}},{"id":57,"attributes":{"feature":true,"title":"iQiyi goes cloud-native with Apache ShardingSphere & Database Mesh","content":" \n \nIn May of this year, we at [SphereEx](https://www.sphere-ex.com/en/) proposed the [Database Mesh 2.0](https://medium.com/faun/database-mesh-2-0-database-governance-in-a-cloud-native-environment-ac24080349eb) concept. \n[Database Mesh](https://www.database-mesh.io/) is a dynamic concept that is constantly evolving, focusing on database traffic governance and providing sharding, load balancing, observability, and audit capabilities based on database protocol awareness. These capabilities address some of the traffic governance issues. \nFurthermore, Database Mesh emphasizes the development of database reliability engineering (DBRE), providing easier-to-use and superior database governance capability. \n \n[iQiyi](https://www.crunchbase.com/organization/iqiyi) shares our view for the concept and vision of Database Mesh, which is to **achieve high-performance database expansion while tackling data governance issues in the cloud**. \n\niQiyi expanded [ShardingSphere-JDBC](https://shardingsphere.apache.org/document/current/en/overview/#shardingsphere-jdbc) based on its business requirements and conducted a series of tests combined with [Pisanix](https://github.com/database-mesh/pisanix), for the implementation of the Database Mesh concept. \n \n### How did iQiyi prepare to go cloud-native with Database Mesh & ShardingSphere-JDBC?\n\nWith the expansion in the number of features, products & service offerings and the surge in the number of users, enterprises have diversified the number of promotional activities to engage and retain users (flash sales, events, etc.). In successful cases, this led to a huge amount of traffic putting great pressure on their databases. \n\nAs a result, enterprises encounter database issues such as secondary delays and slow queries, with some operations failing to meet business requirements. \nMicroservices and cloud-native bring new possibilities for the business roll-out process and governance. But with more diversified business scenarios and stovepipe data application schemes, data control tends to be isolated. Tech teams face problems such as difficult technology selection, high costs, and complicated management and control.\n \nParticularly, cloud-native architecture is growing mature, and the relationship between business applications and database infrastructure is changing gradually. iQiyi hopes to grasp this new trend and adopt unified management to expand and update databases, thus supporting more businesses and applications migrating to the cloud.\n \nTo meet the requirements for database performance and availability in cloud environments, iQiyi needed to migrate [ShardingSphere](https://shardingsphere.apache.org/)'s local distributed capability to the cloud. To achieve this, iQiyi was looking for a tool that can unify cloud database traffic access in a cloud-native environment and achieve the unified and efficient management of cloud traffic and data.\n \nWhile investigating and testing Pisanix, a Database Mesh solution provided by SphereEx, iQiyi also redeveloped ShardingSphere-JDBC, to meet the requirements for sharding, load balancing, configuration and storage, and security when accessing businesses to database governance platforms.\n \n#### 1. Preparation: iQiyi transforms ShardingSphere-JDBC\n\nCurrently, iQiyi uses a unified config center to store database connection configuration. [KMS technology](https://kms-technology.com/) is used to encrypt database access configuration and ShardingSphere-JDBC is used to implement sharding and load balancing. The complete architecture is shown below.\n \n\n\n \nWhen businesses are connected to the data governance platform, they apply for relevant connection configuration. After they are transformed and the access information is encrypted via KMS, they are stored in the unified configuration storage center. When the application starts, the transformed ShardingSphere-JDBC fetches the configuration and monitors configuration changes to support hot configuration updates.\n \nBefore the transformation, when there was a need to change the configuration, scale-out sharding clusters, upgrade cluster version, or migrate a database to the cloud, it usually required the release of a new version. Also, the DevOps teams had to design complicated procedures such as switchover, rollback, timing selection, grayscale traffic, and data verification, to account for various scenarios.\n \nAfter the transformation, the customized ShardingSphere-JDBC can support sharding cluster scaling or binding changes when adding or modifying table sharding configuration. In the configuration center, you can perform visual operations to modify configurations or bind clusters, and select the configuration of reload timing. When the SDK receives the latest configurations, it starts asynchronous tasks to close the old connection pool and replace the existing one. This facilitates the smooth migration of read/write traffic and greatly simplifies the migration of data governance capabilities to the cloud environment.\n \niQiyi plans to introduce Pisanix-Proxy by accessing Database Mesh, further sinking data governance capability from [SDK](https://en.wikipedia.org/wiki/Software_development_kit) to Sidecar. \n \n#### 2. Data governance capability with Sidecar and building a unified data governance based on Pisanix\n\nIn the traffic access layer, as cloud-native applications move closer to microservices and Serverless, users need to configure complex routing rules, support multiple application-layer protocols, and ensure service access security and the observability of traffic. In response to these requirements, iQiyi used middleware to manage [Redis](https://redis.io/) and [MySQL](https://www.mysql.com/) at the very beginning.\n\n\n \nAdditionally, SmartJedis provided a unified configuration center to support iQiyi's hybrid cloud deployment. In the unified configuration center, configurations in different environments could be dynamically supported. In a non-mesh environment, a direct connection is adopted; while in a mesh environment, RedisProxy in Envoy is used to manage Redis protocol traffic and support hot updates of connection configuration, avoiding downtime after Redis is moved to the cloud. \n \nIn terms of MySQL, iQiyi's R&D team tested Pisanix, the specific implementation of Database Mesh. Written in [Go](https://go.dev/) and [Rust](lhttps://www.rust-lang.org/) for the [Kubernetes](https://kubernetes.io/) environment, Pisanix currently supports [MySQL](https://www.mysql.com/). It includes three components: Pisa-Controller, Pisa-Proxy, and Pisa-Daemon, which provide a local database for users and applications. It supports multi-protocol pluggable architecture, shields the status of real data sources, and provides unified database traffic control capabilities for data DevOps teams.\n\nCurrently, iQiyi still uses ShardingSphere-JDBC to support Java applications. Once Pisanix will be further implemented by iQiyi, the company will implement standardized automatic database maintenance via Pisanix, and achieve the cloud-native orchestration of multiple database governance behaviors by supporting multi-language applications. Based on Database Mesh's standard `CustomResourceDefinition`, such as unified database access declaration configuration and programmable database access resource limitations, iQiyi can rapidly achieve the governance and orchestration of cloud-native databases.\n \n#### 3. iQiyi's plan for Pisanix\n\n**1）Data sharding: achieve high performance on par with ShardingSphere-JDBC in the cloud**\n\nData sharding is an effective way to deal with massive data storage and computing, which is why iQiyi chose Pisanix for cloud-native and non-Java scenarios. Data sharding mainly includes four modules: SQL parser, SQL rewriting, SQL router, and result merger.\n \nTo facilitate the migration of ShardingSphere's powerful local sharding capability to the cloud, Pisanix provides data sharding governance capabilities in the cloud based on the underlying database, allowing users to achieve horizontal scaling computing through Pisanix. At the same time, more custom metrics are available to achieve intelligent, stable, and advanced auto-scaling for Pisa-Proxy.\n \nBased on the Pisa-Controller plane, iQiyi can achieve the management and control of data plane components. Pisa-Proxy can also be combined and deployed in the same Pod with business applications in Sidecar mode to monitor MySQL protocol and obtain the traffic of applications accessing the database. Pisanix also provides iQiyi with a variety of governance capabilities:\n- **SQL traffic governance:** achieve multiple load balancing strategies and current limiting by paring SQL.\n- **Access control:** achieve fine-grained permission control based on the relationship between users and data permission.\n- **SQL firewall:** prevent high-risk SQL from executing.\n- **Observability:** expose various database access metrics such as throughput and latency.\n \nFrom iQiyi's point of view, Pisanix enables the high-performance sharding of both Java and non-Java services in the cloud environment. This achievement lays the foundation for the smooth transition of more businesses.\n \n**2）Read/write splitting: increase database throughput.**\n\nTo improve throughput and availability, many systems adopt a primary-secondary database architecture configuration mode, which is a bit complicated. Therefore, when read requests outnumber write requests, read/write splitting should be used to overcome the performance bottleneck of the database in real-world application scenarios.\n \nRead/write splitting is a widely used technical solution to improve throughput in primary-secondary scenarios, and is capable of improving query performance and reducing server load. It also brings the same problem with data sharding, which makes it more complicated for DevOps teams to operate databases.\n \nCurrently, iQiyi evenly distributes query requests to multiple data copies through the configuration mode of one primary and multi-secondary, which improves the processing capability of the system. This method improves throughput and the availability of the system - even when a database breaks down or a disk is physically damaged, the system can still maintain normal functioning.\n \niQiyi plans to adopt Pisanix's dynamic read/write splitting feature to manage multi-primary and multi-secondary database clusters. After connecting to Pisanix, iQiyi will be able to transparently manage the primary/secondary database with read/write splitting so that users can use the database with the primary/secondary architecture just like a monolithic database.\n \n### Future plan\n\nCurrently, iQiyi has completed its internal transformation for ShardingSphere-JDBC. In the future, it plans to combine Pisanix and ShardingSphere to achieve the unified governance of MySQL. \n\nDriven by the ShardingSphere and Database Mesh communities, Pisanix will continue to develop cloud solutions to meet various usage scenarios, with SphereEx providing reliable technical support for iQiyi and accelerating the transition speed to the cloud. \n \nPisanix is a very young project, which means there are some shortcomings. iQiyi's test shows that Pisanix is limited in its expression support for database and table sharding, and its special configuration for SQL needs to be further improved. \n\nNext, the community will focus on improving Pisanix's online capabilities, including operating status visualization, metrics, circuit breaker degradation strategy, and tracing. \nAdditionally, SQL audit, Pisa-Controller's merge with [Istio](https://istio.io/), and other issues related to compatibility and performance have also been put on the agenda.\n \nIn the coming future, iQiyi will build a MySQL-based unified data access standard and solution based on ShardingSphere-JDBC and Pisanix that is still evolving under the Database Mesh concept. \n\nThrough a unified configuration center and customized Sidecar, iQiyi will gradually make the database access details fully transparent to developers. This way, it can simplify the operating process while enhancing the security of database access, simplifying moving applications moving to the cloud.\n \n----------------------------\nFor more information about Database Mesh and Pisanix, follow the links below:\n[Database Mesh 2.0: Database Governance in a Cloud Native Environment](https://medium.com/faun/database-mesh-2-0-database-governance-in-a-cloud-native-environment-ac24080349eb)\n[Pisanix is Available! An Open Source Database Mesh Solution Launched by SphereEx](https://www.sphere-ex.com/news/43/)\n[ Pisanix GitHub](https://github.com/database-mesh/pisanix)\n","date":"2022-12-16","author":"SphereEx","excerpt":"iQiyi migrates ShardingSphere’s local distributed capability to the cloud with Database Mesh’s Pisanix\n\n","createdAt":"2022-12-15T06:58:49.152Z","updatedAt":"2023-01-16T03:48:29.037Z","publishedAt":"2022-12-16T08:22:35.305Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":457,"attributes":{"name":"iQiyi & Database Mesh.png","alternativeText":"iQiyi & Database Mesh.png","caption":"iQiyi & Database Mesh.png","width":1800,"height":766,"formats":{"thumbnail":{"name":"thumbnail_iQiyi & Database Mesh.png","hash":"thumbnail_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":245,"height":104,"size":23.87,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_i_Qiyi_and_Database_Mesh_dc1577e484.png"},"large":{"name":"large_iQiyi & Database Mesh.png","hash":"large_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":1000,"height":426,"size":142.16,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/large_i_Qiyi_and_Database_Mesh_dc1577e484.png"},"medium":{"name":"medium_iQiyi & Database Mesh.png","hash":"medium_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":750,"height":319,"size":105.4,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/medium_i_Qiyi_and_Database_Mesh_dc1577e484.png"},"small":{"name":"small_iQiyi & Database Mesh.png","hash":"small_i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","width":500,"height":213,"size":63.66,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_i_Qiyi_and_Database_Mesh_dc1577e484.png"}},"hash":"i_Qiyi_and_Database_Mesh_dc1577e484","ext":".png","mime":"image/png","size":134.2,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/i_Qiyi_and_Database_Mesh_dc1577e484.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-12-16T07:14:02.426Z","updatedAt":"2022-12-16T07:14:02.426Z"}}},"localizations":{"data":[]}}},{"id":56,"attributes":{"feature":true,"title":"SphereEx builds a complete enterprise data security ecosystem","content":"Guaranteeing data security is essential, to avoid putting users' personal data at risk of leaks, and avoid damage to enterprises' business security and brand reputation. \n\nAlthough data security cannot generate direct monetary returns, it has become crucial for enterprises. Regulations on data protection have been introduced all over the world, making data security of paramount importance.\n\nFrom an enterprise perspective, the challenges faced in data security can be attributed to both internal and external factors.\n\nEnterprises must speed up building their data security systems since they have a limited amount of time to comply with safety regulations. \n\nLarge enterprises and digital-first companies are typically the focus of regulators when it comes to data security regulation. Additionally, if a company has a presence in the EU for example, it must also adhere to GDPR regulations - making building a data security system of the utmost importance.\n\nHowever, Rome wasn't built in a day. There are many pain points in terms of technology and standards when it comes to data security:\n\n- High business transformation costs: WMS (warehouse management systems) are diversified and large in scale, so application transformation entails high costs. \n\n- High risk during the release phase: there's a high risk when switching applications.\n\n- Switching costs: business switchover is challenging, requiring custom-made strategies.\n\n- Scattered data without unified standards: enterprise data is scattered and without unified authority control.\n\n#### Background\n\n**Data security is positively correlated with business coupling**\n\nAccording to regulations, data related to users' security or commercially sensitive data needs to be encrypted. \n\nHowever, traditional data encryption solutions such as hard disk encryption, file encryption, database TDE encryption, database encryption gateway, and application encryption show a very close positive correlation between their data security and business coupling.\n\n\n\nAs mobile Internet takes over, business adjustments and feature launch frequency are rising, with product features and business scope expanding as well. Enterprises must react rapidly to market demands and sophisticated operating models. \n\nAs a result, traditional stovepipe architecture gradually gave way to microservice architecture - making the \"more secure data leads to more coupled business\" assumption obsolete.\n\nIf we pursue low business coupling, we have to sacrifice a certain degree of data security, which is unacceptable to both the internal requirements of enterprises and external industry standards.\n\n**Business scenarios associated with data encryption**\n\nDepending on specific industry requirements, DevOps teams must maintain a set of encryption and decryption systems for real-world business scenarios. \n\nThe self-maintained encryption system often needs to be rebuilt or modified when the encryption scenario changes. Additionally, for services that have already been launched, it's complicated to transparently and securely implement seamless encryption and transformation without modifying the business logic and SQL.\n\nIn terms of new services, data encryption is required. DevOps teams must achieve data encryption based on encryption requirements since everything is new. Rapid business growth, however, makes it difficult for the original encryption strategy to match the new demands. As a result, large-scale business system transformation is required, causing huge upgrading costs.\n\nFor mature services that are already online and are stored in plain text, when it comes to the the migration and encryption (data cleansing) of the old data and the related business, SQL transformation is required - which is quite complicated. \n\nMoreover, the core business needs to be transformed without impacting the service level. The transformation involves establishing a pre-release environment and coming up with a rollback strategy, which will create significant costs.\n\n### SphereEx-DBPlusEngine: A Comprehensive Data Security Solution\n\nIn response to these issues, SphereEx-DBPlusEngine provides an enterprise cross-platform data security solution for heterogeneous environments requiring zero changes to the original code. \n\nIt also provides online data cleansing, custom algorithms, multiple key management (cloud management is also included), and more, to empower enterprises in coping with various data security requirements.\n\nFollowing the launch of cloud key management, encryption, and online data cleansing features with November's [version 1.2.0 release](https://www.sphere-ex.com/news/50/), SphereEx now completes its data security solution with regulation-compliant testing tools and cryptographic computing in the data flow process, establishing a streamlined enterprise-grade data security system.\n\n---\n\n#### 1. One-Stop Security Compliance\n##### 1.1 Security compliance testing tools\n\nEnterprises must determine which data needs to be encrypted, which comes with its own set of challenges as it is difficult to take into account all the legal and regulatory encryption requirements. \n\nLegal and regulatory encryption requirements are fragmented to say the least, as they vary by location. Nevertheless, enterprises need a tool to quickly determine which data needs to be encrypted.\n\nWith this in mind, we introduced our security compliance testing tools. The tools can examine business data in accordance with national standards and overseas laws and regulations (such as GDPR), and automatically detect the fields in the system that need to be encrypted - reducing negative business impact.\n\n##### 1.2 No-code implementation\n\nWhen it comes to data encryption, enterprises are most concerned about applications being changed. Code changes imply cost, stability, and security concerns as well as many unintended risks.\n\nThe open-source project [ShardingSphere](https://shardingsphere.apache.org) developed a mature no-code implementation capability for data encryption. \n\nThis feature has been enhanced by SphereEx-DBPlusEngine. Enterprises can use SphereEx-DBPlusEngine without changing any application or source code, thus avoiding the business risks caused by code modification. SphereEx-DBPlusEngine enables enterprises to quickly implement data encryption requirements to ensure rapid deployment.\n\n##### 1.3 Key management\n\nAs more businesses are transitioning to the cloud, business data naturally run in cloud environments. However, in a public cloud environment, if enterprises still use the original management method when using SphereEx-DBPlusEngine, hidden dangers in terms of security could manifest:\n\n- Encryption is needed for data storage and use in the cloud, as well as during data transfer.\n\n- The management term of the encryption/decryption key is the entire lifecycle of the data. If the key is lost before the data is destroyed, the data cannot be decrypted.\n\nIn order to address the two issues above, SphereEx-DBPlusEngine offers a cloud-based key management approach by abstracting key management as a standard SPI for cloud vendors like [AWS](https://aws.amazon.com/) and [Alibaba Cloud](https://www.alibabacloud.com/).\n\n\n\nTake AWS as an example. When the program initializes the encryption algorithm, it connects to AWS to retrieve the relevant key stored there and then stores the key in the algorithm. \n\nThe entire data encryption process doesn't include any network communication with the cloud, preventing data flow caused by interaction and fundamentally ensuring key security.\n\nBy offering a cloud-based key management solution, SphereEx provides enterprises with incredibly high key management flexibility and improves the convenience and security of the entire encryption system. \n\nIt can also seamlessly interface with each cloud's key management features to offer the best protection. Moreover, SphereEx-DBPlusEngine supports a number of key management methods to interface with cloud-based, public, and private key management.\n\n##### 1.4 Encrypted data cleansing, backwashing, and rewashing\nWhen enterprises need to migrate new services, they often need to encrypt a large amount of new business data to comply with regulations and internal compliance requirements in terms of data security. A traditional encryption method would not only increase the workload but also delay the entire migration process, affecting the business deployment process.\n\nCurrently, DBPlusEngine already provides an encryption solution. For new tables and services, we can directly configure them using encryption rules; but for existing data tables, the plaintext fields in these tables should be cleaned and converted to encrypted content.\n\n\n\nThe data cleansing job is triggered by [DistSQL](https://shardingsphere.apache.org/document/5.1.0/en/concepts/distsql/). Once the program receives the request, it will create a data cleansing job according to the current data cleansing rule and encryption rule. \n\nThe job is divided into two sections: the query and update tasks:\n\n- The query task is responsible for querying the user's table data and retrieving the plaintext fields that need to be encrypted and then pushing them to the channel. \n\n- The update task obtains the data from the channel, encrypts it, and updates it. \n\nThe whole task creation and execution process interacts with the governance center, allowing users to query its progress or clean up the job through DistSQL.\n\nFurthermore, in an OLAP scenario, DevOps teams cannot analyze the encrypted data, while the business must maintain the encrypted state. \n\nIn this context, the decrypt() function can be used to obtain the plaintext data directly without having to backwash the data, allowing your teams to analyze the ciphertext data and obtain the data value.\n\nSphereEx-DBPlusEngine also supports backwashing and rewashing in the following two scenarios:\n- **Backwashing for business data rollback**\n\nIf some data does not need to be encrypted once the business goes online, or when data masking is performed on data that has been encrypted in large batches, it is necessary to backwash the encrypted data and uniformly convert it to plaintext again.\n\n- **Rewashing for key replacement**\n\nIf the key needs to be changed on a regular basis or at a critical point to ensure long-term data security, it is necessary to backwash the encrypted data, convert it to plaintext, and re-encrypt the data using the new encryption method.\n\n#### 2. Compatibility & Flexibility\n##### 2.1 Flexible encryption algorithm\nSphereEx-DBPlusEngine supports complete data lifecycle security management, with particular attention to the encryption capability for data storage security. It is possible to store and access encrypted data without modifying the application side by implementing data encryption on the client.\n\nSphereEx-DBPlusEngine provides customization capabilities in terms of key management methods and support for [IDEA](https://en.wikipedia.org/wiki/International_Data_Encryption_Algorithm) and other encryption algorithms to meet the wide range of data encryption needs.\n\nTo further increase the efficiency of encrypted storage and computing, SphereEx-DBPlusEngine can work with security hardware for complete and high-performance encryption. It can also provide standard security equipment with integrated hardware and software, further lowering the user's threshold.\n\n##### 2.2 Fine-grained encryption capability\nSphereEx-DBPlusEngine supports multi-dimensional and fine-grained data encryption capability, which can implement data encryption at the row and column levels, and then support data encryption at both the user and tenant levels. \n\nAccording to encryption granularity, different encryption algorithms and key management can be flexibly configured to achieve accurate and adaptable data security protection.\n\n##### 2.3 Suitable for private, public, and hybrid cloud environments deployment\n\nTo increase data security, many enterprises distribute all their data across various environments. This is especially true for industries or application scenarios that have strict requirements for data security. They often need to take into account their diverse deployment environments and complex data security environments.\n\nSphereEx-DBPlusEngine can be flexibly deployed in private, public, and hybrid cloud environments to meet various users' needs. Its key management, compliance detection, data cleansing, fine-grained encryption, encryption algorithm adaptation and other capabilities fully satisfy users' needs for data security in hybrid environments, while shielding the differences created by different underlying environments and ensuring a consistent user experience.\n\n### About SphereEx-DBPlusEngine\n\nSphereEx-DBPlusEngine, a database enhancement engine, adopts a pluggable architecture with functional modularity. In addition to data storage, it also provides data sharding, distributed transactions, data security, and other database application architecture enhancement capabilities.\n\nIn November, SphereEx-DBPlusEngine's version [V1.2.0](https://www.sphere-ex.com/news/50/) was released, adding cloud-based key management and data cleansing capabilities for data security. \n\nIt provides enterprises with comprehensive and powerful compliance testing tools, cloud-based key management, encryption and decryption, and cryptographic computing capabilities, further enhancing the data security protection capability of SphereEx-DBPlusEngine.\n\nTo find out more or request a free trial for DBPlusEngine, you can sign up on our website [here](https://www.sphere-ex.com/account/#/login/signIn?redirect=%2F).\n\nAlternatively, if you are an AWS user, you can learn more about our offering on AWS Marketplace [here](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0).\n\n\n","date":"2022-12-07","author":"SphereEx","excerpt":"SphereEx-DBPlusEngine provides an enterprise cross-platform data security solution for heterogeneous environments requiring zero changes to the original code.","createdAt":"2022-12-07T10:36:49.258Z","updatedAt":"2022-12-07T10:58:34.575Z","publishedAt":"2022-12-07T10:58:34.571Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":452,"attributes":{"name":"20221207-184319.png","alternativeText":"20221207-184319.png","caption":"20221207-184319.png","width":2160,"height":828,"formats":{"thumbnail":{"name":"thumbnail_20221207-184319.png","hash":"thumbnail_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":245,"height":94,"size":48.02,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_20221207_184319_85657aa0ba.png"},"large":{"name":"large_20221207-184319.png","hash":"large_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":1000,"height":383,"size":597.12,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/large_20221207_184319_85657aa0ba.png"},"medium":{"name":"medium_20221207-184319.png","hash":"medium_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":750,"height":288,"size":375.15,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/medium_20221207_184319_85657aa0ba.png"},"small":{"name":"small_20221207-184319.png","hash":"small_20221207_184319_85657aa0ba","ext":".png","mime":"image/png","width":500,"height":192,"size":186.88,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_20221207_184319_85657aa0ba.png"}},"hash":"20221207_184319_85657aa0ba","ext":".png","mime":"image/png","size":756.49,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/20221207_184319_85657aa0ba.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-12-07T10:44:03.480Z","updatedAt":"2022-12-07T10:44:03.480Z"}}},"localizations":{"data":[]}}},{"id":55,"attributes":{"feature":true,"title":"We’ve launched on the AWS Marketplace and been recognized as an AWS APN Technology Partner! ","content":"We're excited to kickstart our partnership with AWS! \n\nStarting this week, our products will be [available on the AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0&ref=dtl_prodview-alinfa5ygvic6), and we have officially been recognized as a [Technology Partner](https://partners.amazonaws.com/partners/0018a00001lQQCYAA4) in the AWS Partner Network (APN) global community. \n\nWith this move, we are pleased to team up with AWS to help businesses overcome the challenges and costs of using large amounts of real-world data, managing infrastructure complexities, and achieving their cloud-native transition. \n\nWe're super excited about reaching a broad community of developers through this prominent channel.\n\nBy making DBPlusEngine available free, packaging the popular Apache ShardingSphere project, and providing ShardingSphere for Kubernetes - we're continuing to link data and services simply as well as demonstrating our commitment to open source:\n\n- [SphereEx-DBPlusEngine](https://aws.amazon.com/marketplace/pp/prodview-alinfa5ygvic6) is a distributed computing platform to elastically shard & manage your database on any cloud, built with Apache ShardingSphere at its core. In addition to ShardingSphere's functionality, DBPlusEngine provides features such as autoscaling, traffic governance, enterprise security, high-performance clusters, high-availability clusters, and more. \n\n- [ShardingSphere-Proxy](https://aws.amazon.com/marketplace/pp/prodview-kesvb5m5escpo?sr=0-1&ref_=beagle&applicationId=AWSMPContessa) is a transparent database proxy compatible with MySQL and PostgreSQL, working as a distributed database server to provide data sharding, distributed transactions, read/write splitting, HA, query federation features, and more.\n\n- [ShardingSphere for Kubernetes](https://aws.amazon.com/marketplace/pp/prodview-i34uekoeyemgs?sr=0-2&ref_=beagle&applicationId=AWSMPContessa) uses Helm to install a ShardingSphere-Proxy cluster on Kubernetes and provide HPA and HA capabilities.\n\nThis is the first step for our startup towards promoting the \"[Database Plus](https://www.infoq.com/articles/next-evolution-of-database-sharding-architecture/?itm_source=articles_about_ShardingSphere&itm_medium=link&itm_campaign=ShardingSphere)\" and \"[Database Mesh](https://www.database-mesh.io/index.html)\" development concepts and supporting the world of open source. \n\n\nThis news perfectly demonstrates our hyper-growth mode. In the past 12 months or so, we’ve grown our team, signed multiple new deals & partnerships, and moved to new office spaces. In addition to our AWS partnership, SphereEx has also become a [CNCF member](https://www.cncf.io/about/members/). \n\nWith these partnerships, we look forward to accelerating our growth even further and taking on projects and challenges we love to work on. \n\n\nLearn more about our offering on AWS Marketplace [here](https://aws.amazon.com/marketplace/seller-profile?id=d1a1d3ef-fce8-43d5-a57b-e5b1ec59caf0).","date":"2022-11-30","author":"SphereEx","excerpt":"We’re now an AWS APN Technology Partner, and available on the AWS Marketplace with our DBPlusEngine, and Apache ShardingSphere & ShardingSphere for Kubernetes packaged by us. \n\nThey offer autoscaling, traffic governance, encryption, data sharding, high availability, and DistSQL (Distributed SQL). Available free, for anyone. \n","createdAt":"2022-11-30T06:55:03.268Z","updatedAt":"2022-12-07T10:37:25.697Z","publishedAt":"2022-11-30T07:08:56.019Z","locale":"en","newsType":{"data":null},"cover":{"data":{"id":448,"attributes":{"name":"20221130-143323.png","alternativeText":"20221130-143323.png","caption":"20221130-143323.png","width":2160,"height":828,"formats":{"thumbnail":{"name":"thumbnail_20221130-143323.png","hash":"thumbnail_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":245,"height":94,"size":18.24,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/thumbnail_20221130_143323_1e877e8fe6.png"},"large":{"name":"large_20221130-143323.png","hash":"large_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":1000,"height":383,"size":112.8,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/large_20221130_143323_1e877e8fe6.png"},"medium":{"name":"medium_20221130-143323.png","hash":"medium_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":750,"height":288,"size":76.53,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/medium_20221130_143323_1e877e8fe6.png"},"small":{"name":"small_20221130-143323.png","hash":"small_20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","width":500,"height":192,"size":44.84,"path":null,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/small_20221130_143323_1e877e8fe6.png"}},"hash":"20221130_143323_1e877e8fe6","ext":".png","mime":"image/png","size":86.2,"url":"https://sphereex-media-1305704183.cos.ap-beijing.myqcloud.com/20221130_143323_1e877e8fe6.png","previewUrl":null,"provider":"strapi-provider-upload-s3-compat","provider_metadata":null,"createdAt":"2022-11-30T06:36:47.535Z","updatedAt":"2022-11-30T06:36:47.535Z"}}},"localizations":{"data":[]}}}]}},
"staticQueryHashes": []}