Only new fields get the channel_data_field_# tables?

We considered migrating all the legacy data into the new structure, but in our tests found that with a moderate number of fields and a moderate number of entries, the update could end up at the PHP timeout threshold. In our testing there isn’t much in the way of performance loss in having data in both structures, so we decided to maintain the legacy data structure at this juncture. We have talked about providing post-upgrade instructions for manually migrating this data.

Chiming in for a minute.

Having both options feels like a half-baked solution. For add-ons that target those tables and can’t use the Models (because of intricate queries and whatnot), would now have to support both ways.

I’m sure you’ve all discussed and weighed the pros and cons of the current situation, and in the end the choice is yours to make. But a major version update already is an opportunity to change things drastically, so why not go all the way…?

Happy to dive into this more and hear concerns and options, let’s make sure it comes up at the roundtable tomorrow.

FWIW, when Seth says it could easily exceed the PHP timeout, I’ve seen a handful of vanilla ALTER statements take hours to run from the CLI on databases of unextraordinary size. Our thinking is that we’d rather those folks be able to upgrade to v4 and beyond than having to make the choice to be stuck on v3 or move to a host that will let them run MySQL from the command line and potentially bring their site offline for many hours.

Low, I don’t know if it helps but in my case I added this to my AppHelper and its made a few things easier and I’ve barely had to change queries:

/**
     * @param $fieldName
     * @return string
     */
    public static function getFieldTableName($fieldName)
    {
        if (self::isEE3()) {
            return ee()->db->dbprefix . 'channel_data';
        }

        $fieldId = str_replace('field_id_', '', $fieldName);
        ee()->db->data_cache = [];
        $fieldTableExists = ee()->db->table_exists('channel_data_field_'. $fieldId);

        // Must be a legacy field in EE4
        if (!$fieldTableExists) {
            return ee()->db->dbprefix . 'channel_data';
        }

        // Looks like a new field in EE4
        return ee()->db->dbprefix.'channel_data_field_' . $fieldId;
    }

Actually I suppose that code doesn’t help if you’re fetching a lot of fields at once or have a complicated join.

Yeah, will have to do something similar for Low Search’s Field Search filter anyway, regardless of the migration is done fully or not. Doable, but, well, you know.

You could also take a more hard line approach. If there is a migration option available from EL after the release, just say that Low Search requires that migration. That way you only have to code for and support 1 method of querying data.

Maybe comparable, but when I released Publisher 2 I said I wasn’t supporting Playa and Matrix anymore and customers had to use the migration to Relationship and Grid tool that EL created. Not once did I get negative feedback, which surprised me. I think customers would be understanding of an extra requirement if you wanted to go that route.